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TIES for Dummies 


The TIES method is a forecasting environment whereby the decision-maker has the ability to easily assess and 
trade-off the impact of various technologies without sophisticated and time-consuming mathematical formulations. 
TIES provides a methodical approach where technically feasible alternatives can be identified with accuracy and speed 
to reduce design cycle time, and subsequently, life cycle costs, and was achieved through the use of various 
probabilistic methods, such as Response Surface Methodology and Monte Carlo Simulations. Furthermore, structured 
and systematic techniques are utilized from other fields to identify possible concepts and evaluation criteria by which 
comparisons can be made. This objective is achieved by employing the use of Morphological Matrices and Multi- 
Attribute Decision Making techniques. Through the execution of each step, a family of design alternatives for a given 
set of customer requirements can be identified and assessed subjectively or objectively. This methodology allows for 
more information (knowledge) to be brought into the earlier phases of the design process and will have direct 
implications on the affordability of the system. The increased knowledge allows for optimum allocation of company 
resources and quantitative justification for program decisions. Finally, the TIES method provided novel results and 
quantitative justification to facilitate decision making in the early stages of design so as to produce affordable and 
quality products. 

The steps of TIES: 

1 . Define the problem 

2. Define Concept Space: design and technology concepts identification 

3. Modeling and simulation 

4. Investigate design space 

5. Evaluate of system feasibility/viability: probability of success 

6. Specify Technology Alternatives 

7. Assess Technology Alternatives 

8. Select Best Family of Alternatives 
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The following tutorial explains how to implement each of the steps from a computational or evaluation point 
of view. References associated with the TIES method are listed below and can be obtained from the web site: 
http://www.asdl.gatech.edu/publications/index.html . 

1. Kirby, M.R., Mavris, D.N., “A Method for Technology Selection Based on Benefit, Available Schedule and 
Budget Resources”, SAE 2000-01-5563. 

2. Kirby, M.R., Mavris, D.N., “Forecasting Technology Uncertainty in Preliminary Aircraft Design”, SAE Paper 
1999-01-5631. 

3. Mavris, D.N., Kirby, M.R., ’’Technology Identification, Evaluation, and Selection for Commercial Transport 
Aircraft", 58th Annual Conference Of Society of Allied Weight Engineers, San Jose, California 24-26 May, 1999. 

4. Mavris, D.N., Kirby, M.R., Qiu, S., "Technology Impact Forecasting for a High Speed Civil Transport", World 
Aviation Congress and Exposition, Anaheim, CA, September 28-30, 1998. SAE-985547. 

5. Mavris, D.N., Mantis, G., Kirby, M.R. "Demonstration of a Probabilistic Technique for the Determination of 
Economic Viability," World Aviation Congress and Exposition, Anaheim, CA, October 13-16, 1997. SAE- 
975585. 

So, let’s begin. 

Step 1: Problem Definition 

The first step in TIES is to define the problem in terms of the customer requirements for which the product 
will be designed, the available budget to expend on the development, and the time frame in which the product must 
enter the market. In order to formulate the problem, a customer or societal need must exist or a request for proposal 
must be stated to drive the design of a new product. This need is often termed the “voice of the customer” and is 
typically qualitative, or ambiguous, in nature. For example, a commercial airline performs a market study and identifies 
that a majority of potential passengers wish to have lower fares and more flight time options. These are subjective and 
qualitative “wants” that must be mapped into some economic, engineering, or mathematically quantifiable 
terminology. The result of this step is the identification of the system level metrics which capture the customer 
requirements and will be the measure of success of the system under consideration. 

Table 1: Metrics and Constraints: 


Parameter 

Acronym 

Target or Constraint 

Units 

Performance 




Approach Speed 

Vapp 

= 155 

kts 

FAR Stage II Flyover Noise 

FON 

= 106 

EPNLdB 

Landing Field Length 

LdgFL 

= 11,000 

ft 

FAR Stage II Sideline Noise 

SLN 

= 103 

EPNLdB 

Takeoff Field Length 

TOFL 

= 11,000 

ft 

Takeoff Gross Weight 

TOGW 

= 1,000,000 

lbs 

Economics 




Acquisition Price 

Acq$ 

Minimize 

FY96 $M 

Research, Development, Testing, and Evaluation 

RDT&E 

Minimize 

FY96 $M 

Average Required Yield per Revenue Passenger Mile 

$/RPM 

= 0.10 

FY96 $M 

Total Airplane Related Operating Costs 

TAROC 

Minimize 

FY96 0 

Direct Operating Costs plus Interest 

DOC+I 

Minimize 

FY96 0 
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Step 2: befme Concept Space: Design and_ Technology Concepts 

Once the customer requirements are defined in terms of quantifiable engineering parameters, the thrust of the 
TIES method begins with the definition of the concept space and is driven by innovation and “out-of-the-box” 
thinking. Initially, the experience, knowledge, and intuition of the designer is utilized to identify a potential class of 
vehicles and provides the methodology with a starting point for selecting potential solutions to satisfy the customer 
requirements. The focus of this step is two-fold: identify the space of alternative concepts that is based on a defined 
class of vehicles, and establish the geometric and propulsive design space for which system feasibility is initially 
sought. 

Define Technology and Concept Space 

In the design of any complex system, there exists a plethora of combinations of particular subsystems or 
system characteristics that may satisfy the problem at hand. For example, how many engines are needed? What is the 
cruise speed? What type of high lift system is needed? Is a horizontal stabilizer preferred over a canard? A functional 
and structured means of decomposing the system and identifying component options is through the use of a 
morphological analysis. The Morphological Matrix is formed by identifying the major functions or characteristics of a 
system on the vertical scale and all the possible alternatives for satisfying the characteristics on the horizontal scale. In 
essence, this is where the technology alternatives, both mature and immature, to be considered in later steps are 
defined. Once the matrix is populated, an alternative design concept is defined as a mix of the characteristic 
alternatives. All possible design alternative combinations define the alternative concept space. In general, one 
alternative concept is established to begin the feasibility investigation and will be called the baseline concept and is 
typically drawn from mature technologies. Please refer to references 3 and 4 for more info. 


Table 2: Morphological Matrix for HSCT 


Config 

^^^^^Ahematives 

Characteristics'^^ 

1 

2 

3 
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Vehicle 

C^Wing & TaiT) 

Wing & Canard 

Wing, Tail & 
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Cylindrical 

C^Area Ruled^ 

Oval 


Pilot Visibility 

Synthetic Vision 
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^Conve nti onaT& 
^Nose Drootd 


Mission 

Range (nmi) 

d 5000 d 

6000 

6500 


Passengers 
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dT 300 
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Mach Number 
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2.2 

CT 2.4 

2.7 

Propulsion 

Type 

C^MFTF^) 

Turbine Bypass 
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Fan 
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CCon venti onak^ 

High T Comp 



Combustor < 
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Define the Design Space 

Once the baseline concept is defined from the alternative concept space, the baseline may be further 
decomposed into product and process characteristics. This can be performed via the Morphological Matrix or through 
brainstorming sessions with IPTs. Primary product attributes include the physical design parameters that describe a 
characteristic of the system. In conceptual and preliminary aircraft design phase, all of the design parameters should 
not be fixed but should vary within some specified range until such time as a configuration is “frozen”. The process 
attributes include certification, manufacturing, economic, and operational parameters, which are inherently uncertain. 

Within the context of TIES, the product attributes are the key design variables (with associated ranges) which 
define the design space of interest for a given alternative concept. These design variables are often referred to as 
“control” factors, or variables that are within the designer’s control. These key design variables, and associated ranges, 
define the design space in which system feasibility is sought. The design variable ranges are chosen such that the 
largest possible deviations in the given baseline configuration may be captured. This implies that the system must have 
a converged solution, that is, be capable of flying the specified mission. However, care should be taken so that a 
handful of variables do not artificially dominate the design space due to larger relative ranges. For example, if one 
variable is allowed to deviate ±5%, other variable deviations should be the same order of magnitude. 

Table 3: Design Variables and Ranges With Baseline Configuration. 


Variable 

Minimum 

Maximum 

Baseline 

Value 

Units 

Description 

SW 

7500 

9000 

900 

fa 

Wing Area 

TWR 

0.29 

0.33 

0.29 

~ 

Thrust-to-weight ratio 

TIT 

3000 

3400 

3000 

°R 

Turbine Inlet Temperature 

FPR 

3.5 

4.5 

4.5 

~ 

Fan Pressure Ratio 

OPR 

18 

21 

18 

~ 

Overall Pressure Ratio 

CLdes 

0.08 

0.12 

0.1 

~ 

Design Lift Coefficient 

X2 

1.54 

1.69 

1.609 

~ 

FE kink x-location* 

X3 

2.1 

2.36 

2.36 

~ 

FE tip x-location* 

X4 

2.4 

2.58 

2.58 

~ 

TE tip x-location* 

X5 

2.19 

2.37 

2.19 

~ 

TE kink x-location* 

X6 

2.18 

2.5 

2.18 

~ 

TE root x-location* 

Y2 

0.44 

0.58 

0.51 

~ 

LE kink y-location* 

t/c root 

3 

5 

4 

% 

Wing root thickness-to-chord ratio 

t/c tip 

2 

4 

3 

% 

Wing tip thickness-to-chord ratio 


400 

700 

550 

fa 

Horizontal Tail Area 


350 

550 

450 

fa 

Vertical Tail Area 


* Variable normalized by wing semi-span 
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Step 3: Modeling and_ Simulation 

A modeling and simulation (M&S) environment is needed to facilitate rapid assessments with minimal time 
and monetary expenditures of the alternative concepts identified in the Morphological Matrix. Most companies have an 
in-house developed M&S environment to perform the design trades. The TIES method is not code specific, but the 
M&S tool utilized must have some basic features as outlined in Table 4. One cannot underestimate the importance of 
having a cohesive M&S environment. Without this environment, application of the TIES method is arduous. A 
principle requirement for any decision making process is the ability to quantitatively assess the customer requirements 
that drive a design. This can only be achieved through an M&S environment. The requirements for the M&S 
environment in Table 4 are directed towards aircraft analysis codes. However, one may extrapolate the features needed 
for ANY system design code. 

If the class of vehicle that you are considering falls in the validity range of the analysis tool, you are ready to 
go. Most of the existing public domain codes are based on historical data for evolutionary concepts. If the designs of 
interest fall within this range, the sizing and synthesis codes can accurately assess the objectives. Yet, for a 
revolutionary concept the validity of the results will be questionable. This inability can be overcome through direct 
linking of more physics-based analytical models, or using metamodels to represent the physics-based analysis tool. 
Look at reference 4 for more information. 


Table 4: Required Features Needed for an M&S Environment 


Feature 

Importance 

Purpose 

Parametric inputs 

High 

To quantify outputs in terms of inputs and 
facilitate the use of Response Surface Methods 

Physics based 

Medium High 
(based on level of 
confidence desired) 

To analyze and model evolutionary or 
revolutionary concepts 

Synthesis capability 

Average 

(could use table look- 
ups created off-line) 

To quantify the various disciplines 
(aerodynamics, structure, and propulsion) for a 
given configuration 

Unconstrained mission analysis 

Medium High 

To “size” the system in terms of a fuel and 
thrust balance to fulfill a given mission that 
results in a “sized” vehicle and corresponding 
weights in an unconstrained manner so as to 
employ the use of metamodels for a continuous 
design space 

Robust input definition 

High 

To allow for a wide range of configurations or 
missions to be analyzed 

Economic analysis 

High 

(assumes economics are 
a key driver) 

To immediately quantify the impact of design 
changes on the economic requirements of the 
system 

Responses are quantifiable 

Medium High 

To functionally relate the responses of interest 
to the variations of inputs 

Disciplinary technical metric 
impact factors 

Very High 

To simulate the discontinuity associated with 
the addition of new technologies 

Automation capability 

Average 

To facilitate probabilistic design methods and 
to have a “wrapper” around the tool 

Rapid Assessments 

Average 

To facilitate reduced cycle time 

Access to source code 

Average 

To modify fidelity deficiencies of different 
disciplines as needed and understand internal 
control laws or to add technical metric “k” 
factors 
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Table 5: HSCT Baseline Metrics 


Parameter 

Acronym 

Baseline Value 

Performance 



Approach Speed 

Vapp 

154.1 kts 

FAR Stage II Flyover Noise 

FON 

112.3 EPNLdB 

Landing Field Length 

LdgFL 

9,063.2 ft 

FAR Stage II Sideline Noise 

SLN 

111.6 EPNLdB 

Takeoff Field Length 

TOFL 

12,407 ft 

Takeoff Gross Weight 

TOGW 

937,108 lbs 

Economics 



Acquisition Price 

Acq$ 

218.58 FY96 $M 

Research, Development, Testing, and Evaluation 

RDT&E 

16,124.9 FY96 $M 

Average Required Yield per Revenue Passenger Mile 

$/RPM 

0.1236 FY96$ 

Total Airplane Related Operating Costs 

TAROC 

5.948 FY96 0 

Direct Operating Costs plus Interest 

DOC+I 

5.058 FY96 0 


Step 4: Design Space Exploration 

For the purpose of this tutorial, I will assume that you will use Response Surface Equations (i.e., a 
metamodel) representation of your metrics in conjunction with a Monte Carlo Simulation. In this step, you are trying to 
determine the metric values for any combination of design variables, i.e., where the metrics are as a function of design 
variables relative to the target values you identified in Step 1 . The first step is to create a Design of Experiments (DoE) 
table. DoE is a technique to study the interactions between the design variables and their effects on the response 
metrics. Full factorial DoEs can only handle up to 16 variables, because the number of cases to run increases 
exponentially with more variables. For example, if you had 12 variables with two possible settings or levels, you 
would have 4096 or 2 12 combinations to investigate. This is why it is important to use a fractional factorial DoE or to 
perform a screening test to eliminate some of the non-contributing variables. DoEs for more than 16 variables do not 
exist. Based on the Pareto Principle, it is rare that more than a handful of variables actually contribute to the response 
of interest. You can do a screening test at anytime, no matter how many variables you have. So, let’s perform a 
screening test. 


Screening Test 

Start up JMP 

Go to the JMP Starter window. If the window doesn’t come up when you start JMP, then go to View and 
select JMP Starter and the window below will come up. 
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To create a Design of Experiments (DoE) table for a screening test go to DOE tab 


<$► JMP - JMP Starter 


File Edit Tables Rows Cols DOE Analyze Graph Tools View Window Help 

J] i □ G? U m & ^ m @ [jpT ? ® ® ± P P + J| | (No Data Tables) ^ 



Ready 


NUM 


Then select the Screening Design button and add the number of continuous variables that you are 
considering. Let’s add 22 variables. 
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JMP will add XI through X22 in the window that pops up. The variable ranges are set at a minimum of “-1” 
and a maximum of “1”. If you would like to change the names of the variables from XI to something more intuitive, 
just double click on XI and enter the variable name. Also, if you would rather look at a dimensional DoE, double click 
on the “-1” or “1” and add in the real values. Once you are done, click the Continue button. 


File Edit Tables Rows Cols DOE Analyze Graph Tools View Window Help 

||r* @ I (No Data Tables) 



j Continuous 

ja 2-Level Categorical 
Add JO 3-Level Categorical 


Add 


Remove Selected 


Name Role Values 

Continuous 

-1 

1 ll 

© X2 Continuous 

© X3 Continuous 

© X4 Continuous 

© X5 Continuous 

-1 

1 -1 

-1 

1 

-1 

1 

-1 

1 M 


Screening Design 

—Specify Factors 

Add a Continuous or Categorical factor by clicking its button. Double click 
on a factor name or level to edit it. 


Ready 


NUM [SCRL ^ 


In general, the larger number of runs is better, so choose a Fractional Factorial with a Resolution IV with 
some 2-factor interactions with no value in the block size column and then select the Continue button, or if you have 
messed up for some reason you can hit the Back button and it will take you to the previous screen. 
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The next screen will come up. There is a lot of information here. In particular, under the Change Generating 
Rules you can modify the choice of different fractional factorial designs for a given number of factors. The Aliasing of 
Effects button shows you the aliasing structure of the design you have selected and the Coded Design button shows 
you the pattern of high and low values for the factors in each run. For our purposes, the DoE that JMP will create is 
fine. We do need to add a few things before continuing. In particular, add 1 center point so that any quadratic effects 
could be simulated. And under the drop menu for “Run Order”, select the option “Keep the Same” rather than the 
default of “Randomize” so that you can always repeat the identical DoE in the future. Once you are done, select the 
Make Table button. 



The following window will come up. This is your DoE for your screening test. If you did not enter in real 
values or names for your input variables then the table represents the non-dimensional settings of the 22 variables that 
you have (XI through X22). It shows you that there are 129 rows which corresponds to the number of cases to be 
executed by your analysis code. In addition, the settings for the 22 variables that you have are shown, i.e. “-1” 
corresponds to the minimum setting of the variable, “+1” is the maximum value, and “0” is the midpoint. Note you can 
also change the labels on the columns to reflect the actual variable names that you have. You can do this by double 
clicking on the XI column heading. You can then change the name and tab over to change the rest. 
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Now, what you need to actually run your analysis tool is the setting for the different variables for each case in 
the DoE, i.e., and actual values not non-dimensional ones. If you place the mouse cursor in the cell under “XI” and 
row 1, you can highlight the entire DoE table and then copy it into Excel. Below is shown a sample of a highlight. If I 
were to copy this and then paste it into Excel, I would get all the info for XI through X12 and rows 1 through 19. You 
want the entire DoE table. NOTE: make sure that your copy and paste areas are the same dimensions. If they are not, 
your results will be messed up. 


4$ JMP - [Fractional Factorial] 

H0E 


[£) File Edit Tables Rows Cols DOE Analyze Graph Tools View Window Help - 1 g 1 x| 



JJ| Ki ' <3* ED £ P P + | Fractional Factorial jJ 

' r Fractional Factorial 

Design Fractional Factor ii 
w Model 


Pattern XI 3 

XI 4 

a 

+++++_ 


1 


1 


1 



-1 

-1 

-1 


-1 

-1 

-1 



-1 



2_ 

3 





1 


1 



-1 

-1 

1 


1 

1 

1 


1 

1 

— +_+++++++++.++.._ 


1 


7 


4^ 



-1 

1 

-1 


1 

1 

1 


1 

1 

w Columns (24/12) 

4 

5^ 

6 

7 

8 
9 

++ +---+ 


1 


1 


1 


k 

-1 

1 

1 


-1 

-1 

-1 



-1 


+..++++++++++.+++. 


j_ 


1 


1 




-1 

-1 


1 

1 

1 


1 

1 

© Pattern ^ 

+.+ + .. +++ 


1 


1 


1 



1 

L. 

. 1 


-1 

-1 

-1 



-1 

□ 

□ 

□ : 

□ ■ 

□ 

□ 

□ :• 

□ ■: 

□ ^ 

□ si 

© 

© XI 3 
© XI 4 
© XI 5 
© XI 6 
© XI 7 
© XI 8 
© XI 9 
© X20 
© X21 
© X22 
© Y 




1 


1 


1 



1 

1 


.1 

-1 


-1 



-1 

„„ +++++++++++ — ++ 


j_ 


1 


1 



1 

1 

1 


1 

1 

1 


1 

1 


- + -~ ++++ — + - 



1 


1 



1 




-1 

-1 

-1 

1 



1 

— -1 

iL 

-1 

: 

11 

12 

13 

14 

15 

16 

17 

18 

1 1 Mil 1 II 

— 

7 

— 

1 

— 

1 

— 


-1 

1' 

-1 

— 7 

-1 

-1 

-f 


1 


r 

— 

— 

„_ + . ++++++ „._ ++ . ++++ 


1 


1 


1 



-1 

1 

1 


1 

1 

1 




Put your 
cursor here 

++++--+ 


1 


1 


1 



1 

-1 

-1 


-1 

-1 

-1 


1 


— ++-+++++ — +.++—+ 


7 


1 


1 



1 

-1 

1 


1 

1 

1 




—+++.++++ — ++.+.. 


1 


1_ 


1 



1 

1 

-1 


1 

1 

1 




_. ++++ — +++++++++ . + 



L 



1 



1 





1 

1 

iL 

-1 

-1 

-1 

1 

1 

_ 


— 

1 

, 1 ,, 

— 

1 

1 

— 

1 

1 

1 

1 

— ; 


-1 

-1 

-1 

-1 

-1 

1 

— -J 

1 

-1 

-1 

1 

-1 

1 

— -J 

1 

-1 

1 

1 



19 

..+..+...++..++.H 


1 


1 

1 



-1 

1 

-1 


-1 

1 

1 



1 


20 

^.*+ ++ __ ++ __ ++ _+__ + 


1 


1 

1 



-1 

1 

1 


1 

-1 

-1 


1 

-1 


21 

++..++..+.++. 


1 


1 

1 



1 

-1 

-1 


-1 

1 

1 



1 


22 

„ + . + . +++ .. ++ .. + . +++++ 


1 


1 

1 



1 

-1 

1 


1 

-1 

-1 


1 

-1 


23 



1 


1 

1 



1 

1 

-1 


1 

-1 

-1 


1 

-1 


24 



1 


1 

1 



1 

1 

1 


-1 

1 

1 



1 


25 

--++ — ++++..++++.+. 


1 


1 

1 



-1 

-1 

-1 


-1 

1 

1 


1 

-1 


26 

..++_+++ — ++-++—++ 


1 


1 

1 



-1 

-1 

1 


1 

-1 

-1 



1 


27 



1 


1 

1 



-1 

1 

-1 


1 

-1 

-1 



1 


28 

..++.++..++++ — +.+++ 


1 


1 

1 



-1 

1 

1 


-1 

1 

1 


1 

-1 


29 

..+++..++ ++++.+— 


1 


1 

1 



1 

-1 

-1 


1 

-1 

-1 



1 


'*■ Rows 

30 

„ +++ . + .. ++++ ... + „„ + 


1 


1 

1 



1 

-1 

1 


-1 

1 

1 


1 

-1 


All Rows 129 

Selected 19 

Excluded 0 

Hidden 0 

Labelled 0 

Ready 

31 



1 


1 

1 



1 

1 

-1 


-1 

1 

1 


1 

-1 


32 

-+++++++ — ++ — +-+ 


1 


1 

1 



1 

1 

1 


1 

-1 

-1 



1 


33 

.+ ►..+ 


1 


-1 



-1 

-1 

-1 


-1 

1 

-1 



1 


34 



1 

=3 

-1 

- 


iL 

~1 

1 

iL 

1 

-1 

1 

iL 

1 

-1 


P 

NUM SCRL 


To copy the entire table, highlight all cells as shown below and then go to Edit and then click Copy. The 
entire DoE is in the clipboard and you can go directly to Excel. 


JMP - [Fractional Factorial] 

Edit 


El E* | 

II ■ □_ 


| Tables Rows Cols 



|| ^ ' A P P + |j | Fractional Factorial 7j 

u 

X8 

X9 

XI 0 

I XII 

XI 2 

XI 3 

XI 4 

XI 5 

XI 6 

XI 7 

XI 8 

XI 9 

X20 

X21 

X22 

Y 



-1 

-1 


1 

1 

1 

-1 

-1 

-1 

-1 


-1 

1 

1 

-1 



■ 

1 

1 


-1 

-1 

-1 

1 

1 

1 

-1 


1 

1 

1 

1 


■ 

1 

1 


-1 

-1 

-1 

1 

1 

-1 

1 


-1 

-1 

1 

-1 


i 

-1 

-1 


1 

1 

1 

-1 

-1 

1 

1 


1 

-1 

1 

1 


■ 

1 

1 


-1 

-1 

-1 

1 

1 

-1 

-1 

1 

-1 

1 

-1 

-1 


i 

-1 

-1 


1 

1 

1 

-1 

-1 

1 

-1 

1 

1 

1 

-1 

1 


1 

-1 

-1 

1 

1 

1 

1 

-1 

-1 

-1 

1 

1 

-1 

-1 

-1 

-1 


* -i 

1 

1 

— 

-1 

1 

1 

-1 

-1 

1 

1 

1 

1 

-1 

-1 

-1 




-1 

-1 


1 

-1 

-1 

1 

1 

-1 

1 

1 

-1 

-1 

-1 

1 


7 

-1 

-1 


1 

-1 

-1 

1 

1 

1 

-1 

1 

1 

1 

-1 

-1 


riT=Tl 1 

1 

1 


-1 

1 

1 

-1 

-1 

-1 

-1 

1 

-1 

1 

-1 

1 


g3-i -i 

-1 

-1 


1 

-1 

-1 

1 

1 

1 

1 


1 

-1 

1 

-1 


Li i 

1 

1 


-1 

1 

1 

-1 

-1 

-1 

1 


-1 

-1 

1 

1 


n i 

1 

1 


-1 

1 

1 

-1 

-1 

1 

-1 


1 

1 

1 

-1 


H i 

-1 

-1 


1 

-1 

-1 

1 

1 

-1 

-1 


-1 

1 

1 

1 


y-i -i 

1 

1 


1 

-1 

-1 

-1 

-1 

1 

1 

1 

1 

1 

1 

-1 


B-i i 

-1 

-1 


-1 

1 

1 

1 

1 

-1 

1 

1 

-1 

1 

1 

1 


a i -i 

-1 

-1 


-1 

1 

1 

1 

1 

1 

-1 

1 

1 

-1 

1 

-1 


H i 

1 

1 


1 

-1 

-1 

-1 

-1 

-1 

-1 

1 

-1 

-1 

1 

1 


fl-i 

-1 

-1 


-1 

1 

1 

1 

1 

1 

1 


1 

1 

-1 

-1 


g.i i 

1 

1 


1 

-1 

-1 

-1 

-1 

-1 

1 


-1 

1 

-1 

1 


emI i -i 

1 

1 


1 

-1 

-1 

-1 

-1 

1 

-1 


1 

-1 

-1 

-1 


Efil 1 1 T 

-1 

-1 


-1 

1 

1 

1 

1 

-1 

-1 


-1 

-1 

-1 

1 


ni-i | ~ 

-1 

-1 


-1 

-1 

-1 

-1 

-1 

-1 

-1 


-1 

-1 

-1 

-1 


EEl-i 1 T 

1 

1 


1 

1 

1 

1 

1 

1 

-1 


1 

-1 

-1 

1 



ESI 1 1 -T 

1 

1 


1 

1 

1 

1 

1 

-1 

1 


-1 

1 

-1 

-1 



Efl 1 1 T 

-1 

-1 


-1 

-1 

-1 

-1 

-1 

1 

1 


1 

1 

-1 

1 



ri-i I -T 

1 

1 


1 

1 

1 

1 

1 

-1 

-1 

1 

-1 

-1 

1 

-1 




-1 

-1 


-1 

-1 

-1 

-1 

-1 

1 

-1 

1 

1 

-1 

1 

1 



Eg i -i 

-1 

-1 


-1 

-1 

-1 

-1 

-1 

-1 

1 

1 

-1 

1 

1 

-1 



m 1 1 

1 

1 


1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


HP1I11S 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 






















— 

— 1 

1 
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TIES for Dummies 


You will be given an Excel spreadsheet entitled “convert_screening” which will take the above non- 
dimensional table from JMP and convert the design variables to real values to use in the command line rundoe. Open 
the “convert screening” file and make the active cell CIO. Then go under Edit and select Paste Special and you will 
get the following window, select Unicode Text or Text, and press OK. NOTE: look at the dimensions of the table you 
have in the “convert screening” Excel file and make sure you have the right number of columns for the variables and 
the number of rows for the number of cases. If you have more or less than either one of these, you need to increase or 
decrease the dimensions by copying or deleting the cells or adding more columns, etc before you paste the JMP DoE 
table. 



And you will see that the table you selected in JMP will fill out the table in Excel. What you see here is the 
non-dimensional table with the variables listed above (e.g. Wing area, T/W, TIT, etc.) and the corresponding “real 
value” ranges if you scroll to the right. The minimum value for the wing area is 7500 that corresponds to the “-1” of 
the JMP table. The maximum value is 9000 that is the “+1” value of JMP. You need to input your design variable 
names and ranges as applicable to your problem. Also note that the mid-point (i.e., “0”) is automatically calculated 
from the “-1” and “1” values you entered. 
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Now, if you scroll over to the right, you will see the real number DoE values that were converted from the 
non-dimensional JMP table as shown below. 



You need to copy this tables from cell Z10 (or wherever the cell is that corresponds to the case number) to the 
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TIES for Dummies 


Now, open a new sheet and go to Edit and the Paste Special. And the following window will come up. You 
want to select Values and then OK. 



Your values will be pasted into a 22 by 129 range. Now go to File and then Save As and you will get a “Save 
As” window. Under Save as type, choose the “Formatted Text (Space delimited)” option and call your file “doe. table”. 
Then hit the Save button. 


Save in: [ ~ i 

3 a| M a|Ri«lal 3Jl 

_| Telnet 


jWAC98_HSCT_FIX 


ZlWAC99_HSCT_Tech 



File name: | doe. table 3 

Save as type: [Formatted Text (Space delimited) 


You will get the warning below, just hit OK. 



Then go to File and select Close. You will get another warning as shown below. Select the No option because 
you want the file in the space delimited format, not an Excel format. 



Now you need to FTP this file to your account to run your analysis code. Make sure you FTP as ASCII NOT 
binary. I assume you know how to do this. Then you need to set up your “rundoe” script. An example script for an 
HSCT design space is shown below. You will create one similar to this on your UNIX account. Call the file rundoe. 
Additionally, when you ftp your doe.table file to your account, make sure that there are spaces between each number. 
In our case, we should have 23 columns of numbers and 129 rows. If your file did not transfer as such, go back to 
Excel and copy your table again to a new worksheet and increase the column width so that you see spaces between the 
numbers and then save the file again. 

# ! /usr/local/bin/tcl -f 
for_file line doe.table { 

lassign $line a varl var2 var3 var4 var5 var6 var7 var8 var9 varlO varll varl2 varl3 varl4 varl5 varl6 
puts stdout "######## CASE # $a ########" 

# Set up the input to tsw 

set file [open varfile w] 
puts $file "CONFIN SW $varl " 
puts $f ile "CONFIN TWR $var2 " 
puts $f ile "CONFIN ETIT $var3 " 
puts $f ile "CONFIN EFPR $var4 " 
puts $f ile "CONFIN EOPR $var5 " 
puts $f ile "DESIGN CLDESGN $var6 " 
puts $f ile "DESIGN X2 $var7 " 
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TIES for Dummies 


# 


Run 


puts 

$f ile 

'DESIGN 

X3 

$var8 " 

puts 

$f ile 

'DESIGN 

X4 

$var 9 " 

puts 

$f ile 

'DESIGN 

X5 

$varl0 " 

puts 

$f ile 

'DESIGN 

X6 

$varll " 

puts 

$f ile 

'DESIGN 

Y2 

$varl2 " 

puts 

$f ile 

'DESIGN 

TCRT $varl3 " 

puts 

$ f ile 

'DESIGN 

TCTP $varl4 " 

puts 

$f ile 

'DESIGN 

SHREF $varl5 " 

puts 

$f ile 

'DESIGN 

SVREF $varl 6 

close $file 





tsw 

puts stdout " Running tsw" 

catch "exec tsw -input opt_baseline -output case$a varfile" 
exec cp case$a f 198. in A 

exec f lops_subs_modif ied 

exec mv flopsin.new case$a | 


exit 


Make sure your baseline file 
name that you are using 
corresponds to this name 


Another sample file is shown below. 

# ! /usr/local/bin/tcl -f 
for_file line doe. table { 

lassign $line i varl var2 var3 var4 var5 var6 
puts stdout "######## CASE # $i ########" 

# Set up the input to tsw 

set file [open varfile w] 


puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
set land_wt 
puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
puts $file 
close $file 

# Run tsw 


'MAININ THRSO $varl " 
'TOLIN THRTO\ [ 1 \ ] $varl " 
'TOLIN THRTO\ [ 2 \ ] $varl " 
'TOLIN THRT0\[3\] $varl " 
'TOLIN THRTO\ [ 4 \ ] $varl " 
'TOLIN THRT0\[5\] $varl " 
'TOLIN THRT0\[6\] $varl " 
'TOLIN THRTO\ [ 7 \ ] $varl " 
'TOLIN THRT0\[8\] $varl " 
'TOLIN THRT0\[9\] $varl " 
'TOLIN THRTO\ [ 1 0 \ ] $varl " 
'MAININ GW $var2 " 

[expr ( $var2-381 987 . 4 ) ] 
'MAININ WLDG $land_wt " 
'MAININ CLTOM $var3 " 
'TOLIN CLTOM $var3 " 
'MAININ CLLDM $var4 " 
'TOLIN CLLDM $var4 " 

'TOLIN ALPROT $var5 " 
'TOLIN ALMXLD $var6 " 



This syntax is based on THRTO 
having a 1 0 element array and this 
is how you substitute a particular 
element of that array 


puts stdout " Running tsw" 

catch "exec tsw -input base -output case$i varfile" 


exit 

Now you need to write another script to run your DoE. A simple script is shown below. You can call the file 
anything you like. Just make sure that you change the mode of the file to make it executable. For example, if you call 
the file runcases , then at the UNIX prompt, type chmod 7 00 runcases. Also, do this for the rundoe file you 
created. 


echo "Running the bl script for DoE cases 
i=l 

imax=12 9 

while [ $i -le $imax ] 
do 


echo "Now running file: $i" 

flops case$i case$i.out 
echo "$i completed" 
echo " *********************** 
let i=i+l 


done 


To execute both the rundoe and the runcases , you simply need to type the file name and the script will run. 
You want to run the rundoe first and create all your case files and then run the runcases to execute FLOPS. Just a 
reminder, your baseline file is called opt baseline in the first rundoe script and base in the second. Make sure you have 
the baseline file and the doe. table file in the directory that you are running the script. 
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TIES for Dummies 


Now, you will need to extract the metric data for each case. To do so, you will use the parse98 program. A 
sample file is shown below. I call this file getinfo. You also need to change the mode of the file as you did above. 
NOTE, make sure you are parsing the proper info before you run this script. You can check this by executing a given 
line at the UNIX command prompt. For example, 

parse98 -search "TOGW" -read 3 -occurance 1 -offset 0 easel. out 


# ! /usr/local/bin/wishx -f 

set Number_of_Cases 289 
exec touch summary_total 
exec rm summary_total 

exec touch noise_summary 
exec rm noise_summary 

exec echo " TOGW TOFL LDGFL VAPP ACQ RDTE RPM TAROC DOC+I" » summary_total 
exec echo " THRUST TOGW K2T K2A KlA WLDG" » noise_summary 
for {set i 1} { $i <= $Number_of_Cases } { incr i 1} { 


puts stdout "getting info case$i" 


set 

togw 

[ exec 

parse98 

-search 

"TOGW" 


-read 

3 

-occurance 1 

-offset 

0 

case$i . out ] 

set 

tof 1 

[ exec 

parse98 

-search 

"DFAROFF 

" 

-read 

3 

-occurance 1 

-offset 

0 

case$i . out ] 

set 

ldgfl 

[ exec 

parse98 

-search 

"DFARLDG 

" 

-read 

3 

-occurance 1 

-offset 

0 

case$i . out ] 

set 

vapp 

[ exec 

parse98 

-search 

"DVAPP " 


-read 

3 

-occurance 1 

-offset 

0 

case$i . out ] 

set 

acq 

[ exec 

parse98 

-search 

"Final" 


-read 

7 

-occurance 1 

-offset 

0 

case$i . out ] 

set 

rdte 

[ exec 

parse98 

-search 

"TOTAL RDT&E COST" 

-read 

4 

-occurance 1 

-offset 

0 

case$i . out ] 

set 

rpm 

[ exec 

parse98 

-search 

"Average 

Yield/RPM" 

-read 

4 

-occurance 1 

-offset 

0 

case$i . out ] 

set 

taroc 

[ exec 

parse98 

-search 

"Method 

SubTotal " 

-read 

5 

-occurance 1 

-offset 

0 

case$i . out ] 

set 

doci 

[ exec 

parse98 

-search 

"Method 

SubTotal" 

-read 

4 

-occurance 1 

-offset 

0 

case$i . out ] 

set 

thrust 

[ exec 

parse98 

-search 

"DTHRUST 

" 

-read 3 


-occurance 1 

-offset 

0 

case$i . out 

set 

k2t 

[ exec 

parse98 

-search 

"K2T =" 


-read 3 


-occurance 1 

-offset 

0 

case$i . out 

set 

k2a 

[ exec 

parse98 

-search 

"K2A =" 


-read 3 


-occurance 1 

-offset 

0 

case$i . out 

set 

kla 

[ exec 

parse98 

-search 

"KlA =" 


-read 3 


-occurance 1 

-offset 

0 

case$i . out 

set 

wldg 

[ exec 

parse98 

-search 

"MAXIMUM 

LANDING WEIGHT" 

-read 4 


-occurance 1 

-offset 

0 

case$i . out 


exec echo " $i $togw $tofl $ldgfl $vapp $acq $rdte $rpm $taroc $doci" >> summary_total 
exec echo "$i $thrust $togw $k2t $k2a $kla $wldg" >> noise_summary 


puts stdout "Parsing is now completed!! 
exit 


You should get the TOGW value from case l.out. This command is looking for the character string “TOGW” 
(-search), for the first time it occurs (-occurance 1), looking on the same line that parse98 finds “TOGW” (- 
offset 0), getting the 3 rd character string on that line (-read 3), and will return that value to the screen. Do this 
for each metric to make sure you are getting the right values. All of your data will be extracted and put into a file called 
summary _total. A sample of one of these files is shown below for the first 12 cases. You need to ftp this back to your 
PC and then open in Excel. 


TOGW TOFL VAPP FON SLN ACQ RDTE RPM 

1 1085486.8 14949 182 114.242 113.234 205.049 16850.698 0.13147 

2 919266.1 15533 167.5 114.159 110.943 187.345 15240.811 0.11494 

3 1020994.9 16734 176.5 115.978 112.341 195.047 15806.297 0.12553 

4 937534.3 13356 169.1 109.944 111.32 193.251 16021.355 0.11602 

5 1020319.5 17516 176.4 116.451 112.241 195.275 15820.057 0.12552 

6 955509.4 13322 170.7 110.159 111.527 194.7 16143.683 0.1181 

7 1028392.3 14247 177.1 112.5 112.526 199.759 16433.2 0.12553 

8 860729.5 14323 162 112.585 110.359 181.794 14819.3 0.10916 

9 951218 14090 170.3 112.972 111.292 193.578 16015.582 0.1176 

10 1019691.4 15701 176.4 114.206112.224 194.959 15808.73 0.12532 

11 976201.8 15621 172.6 114.724 111.729 192.515 15641.596 0.12062 

12 1018651.7 14374 176.3 112.924112.345 198.327 16294.583 0.12447 
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TIES for Dummies 


Now, open Excel and go to File, then Open, and then select “List all file types” and find the summary Jotal 
file that you ftp. You will get the following window. You want to open it as a Delimited character file. Select that 
option and hit Next >. 



You will get the window below. You want to open it as delimited by Tab and Space. Then hit Next. 


[Text Import Wizard- Step 2 of 3 EE 


This screen lets you set the delimiters your data contains, You can see 
how your text is affected in the preview below, 


Delimiters 

W lab V~ Semicolon V~ Comma 
!|pacej Other: r 


T^eat consecutive delimiters as one 


Text Qualifier: 


3 


Data preview 



0 

TOGW 

TOFI 

FOH 

BIN 


VAPP 

ACQ 


R 



1 

1085406 . 8 

14949 

114 . 242 

113 . 

234 

182 

205 . 

049 

1 

Zl 


2 

919266 . 1 

15533 

114 . 159 

110 . 

943 

167 . 5 

187 . 

345 

1 



3 

1020994 . 9 

16734 

115 . 978 

112 . 

341 

176 . 5 

195 . 

047 

1 



4 

937534 . 3 

13356 

109 . 944 

Ill . 

320 

169 . 1 

193 . 

251 

1 



5 

1 07031 9 5 

1 751 6 

116 451 

1 1 2 

341 

1 76 4 

1 95 

275 

1 

-1 





You will get the below window. Hit Finish and you^data will be placed into separate cells. 


[Text Import Wizard - Step 3 of 3 


This screen lets you select each column and set 
the Data Format. 

'General' converts numeric values to numbers, date 
values to dates, and all remaining values to text. 


rColumn data format 
(* General 
O Text 

CDate: foY 3 
C Do not import column (Skip) 


IWxl 


Data preview 





Genera 1 


^.Rneral 1 


rroGw 

1085486.8 
919266.1 

1020994.9 
937534.3 
llQ2Q319.5 

rOFL 

14949 

15533 

16734 

13356 

tON 

114.242 
114 . 159 
115.978 
109.944 

mSzu 

BLN 

113.234 
110.943 
112.341 
111.320 
11? ?41 

VAPP 

182 

167.5 

176.5 
169.1 
176 4 

ACQ 

205.049 
187.345 
195.047 
193.251 
1 95 ?75 1 

1 


d 


Cancel | < Back | 
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Now select all the data and copy it back to JMP so that you can look at the influence of each design variable 
on the metrics (or responses) of interest. Highlight all the data and copy. Open JMP and recreate your DoE. You will 
need to add more columns for your data. To do this, go to Cols and then select Add Multiple Columns... and you will 
get the following window. The DoE table has one response column entitled “Y”. For the example I am showing here, 
there are 8 responses. Hence, we need to add 7 columns “After last Column” and then hit OK. 



As you see below, 7 columns were added at the end. You can change the names of the columns just as you did 
with the design variables described previously. 
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Now, highlight your response columns in JMP (as shown below) and go to Edit and then Paste and you will 
see that your response columns fill with the data that you copied from Excel. 



Now you need to analyze your data. Go to Analyze and select Fit Model and the following window will pop 
up. In the top left corner of the window are your variables. Click on your first variable (Wing Area) and it will be 
highlighted. Then scroll down until you see your last design variable (XW). Hold down the shift key and then click on 
the last variable. You want to select the Add button. 
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Your design variables will then be placed into the previously empty area under “Construct Model Effects”. In 
the same manner that you selected your design variables, select your responses and click on the Y button. 


JMP - Fit Model 
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IMPORTANT : Now, go under the drop menu for Model Specification and unclick the “Center 
Polynomials” option. If this option is enabled a continuous term participating in a crossed term will be centered by its 
mean. Thus, your coefficients will not be simple numerical values. So unclick it to avoid difficulties. Once you have 
done that then click the Run Model button. You will need to do this EVERYTIME you analyze your data! ! ! 
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The following window will appear. Your responses will be listed in the order in which you entered them in 
the Fit Model window. In the example below, you will see the “Actual by Predicted Plot”. Each of the little dots in this 
plot represent one of the 129 cases you ran. If you put your mouse over a dot, JMP will tell you which case it 
corresponds to. Also, as you can see below, the dotted red lines correspond to how well you model is being predicted. 
The straight red line is indicative of a perfect model fit. The further away the dotted lines are from this “perfect” line 
implies that your analysis code is NOT being predicted very well by the model you chose. This large deviation is 
expected in a screening test since you are only trying to identify the main contributors to the responses from the use of 
a linear DoE. You are not actually trying to fit an RSE to the analysis code. However, if your analysis code was very 
linear, then the screening test would probably capture the variability quite well. Remember, the screening test is a 
linear model of your responses. 
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Now, we would like to know which of the 22 variables that we are considering actually contribute the most to 
our different responses. We can do this via a Pareto Analysis, which results in a Pareto plot. To view the Pareto plot 
for a response, click the little red T by the “Response TOGW” drop menu and the following will appear. Click on the 

“Effect Screening” option and then select the “Pareto Plot” option. 
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The Pareto plot shows the individual influence of each design variable on the response with the horizontal 
bars, and then the cumulative effect of the variables with the line. As you can see, the T/W and the t/c_tip are the two 
primary contributors to the TOGW, while X5 and Throttle Ratio hardly contribute at all. Here is where you can down 
select to the top 7, 8, or 1 1 main contributors and use those to create your RSEs. Do this for each of the responses and 
then you can identify which variables contribute to all the responses. For computational purposes, you would like to 
select a common set of variables such that you only have to run 1 DoE to capture all of the responses. However, this is 
not always possible. 



You may want to copy the Pareto Plots to PowerPoint for presentation purposes. To do so, select the + icon 
in the menu bar and then click the bar that says “Pareto Plot of Transformed Estimates” or on the Pareto plot itself and 
you will see that the entire section is highlighted as below. Then go to Edit and select Copy and you can paste the 
image in any software you like. 
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Let’s talk a little bit more about how you select the important contributing variables. In particular, based on 
the Pareto Principle, 80% of the variability in a response is due to 20% of the variables. So, for TOGW, one might 
select the variables that are contained in the box below. However, look at the individual effects of the variables that are 
contained in the oval. Each of the variables contribute about the same amount. In fact, their effects could be 
indistinguishable. This is where you need to use your engineering knowledge and experience as to which variables to 
choose. Since you are doing a screening test, you are only looking at linear effects and some variables may show up as 
significant, when in fact they are not. This is also due to the confounding structure of a linear DoE. Please refer to the 
JMP Help for more info on this. So, lesson learned. . .DON’T ARBITRARILY PICK VARIABLES WITHOUT FIRST 
UNDERSTANDING WHAT IS HAPPENING. 
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The Pareto Plot is a means to visually determine the most significant contributors to a response. However, you 
can also determine the important variables numerically. This information is actually provided as part of the fit model 
option. Consider the picture below. Without going into the mathematics behind the generation of the number below, 
let’s put the definition into something more tangible. Under the Parameter Estimates bar, you see each of the variables 
you entered for the main effects of the fit model. Now, look at the two columns entitled “t Ratio” and “Prob>|t|”. The “t 
Ratio” column represents the ratio of the estimate to its standard error, or effectively the signal to noise ratio of that 
given parameter’s influence to the response. The larger the number, the more influence that parameter estimate has on 
the response. You can also determine this by inspection of the “Prob>|t|” value in the next column. As a general rule of 
thumb, if this value is less than 0.05, then the parameter estimate significantly influences the response. If you compare 
the variables that had a value of less than 0.05 in the picture below to the Pareto plot generated above, you will find 
that the variables contained in the box of the Pareto Plot are the same. So, you could identify your significant 
contributors either way. But, if you had an enormous set of variables, you would probably want to choose the Pareto 
Plot to down select. 
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Another important feature of JMP is the Prediction Profile feature as shown below. This should appear 
automatically when you fit your model. If it is not at the bottom of the window, then go to the drop menu at the top 
entitled “Least Squares Fit” and enable the drop menu, then select the “Profilers” option and then select the “Profile” 
option and scroll back down to the bottom of the window. On the left are the responses you have. Each of the bars is 
showing you the influence of a given design variable on the response. You could pick your variables from here, but 
when you get down to ones that “look” to have the same influence, the Pareto Plots help you make better decisions. 
The dotted vertical red lines will move if you left mouse click and drag. The line that you grab and move will change 
the variable setting (value shown in red) and then update the response value for the new variable setting. Play with this 
to get a feel for the Profiler. 




Also with the Profiler, you can change the range of the variables and the responses. In effect, you can zoom in 
on a range or zoom out. You do this by putting your mouse over the name of the variable or the response, hold down 
the “Ctrl” button and then left mouse click. And the window below will come up for a response and the window below 
that will come up for an input variable. 
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That’s pretty much it for the Screening test. For the variables that do not really contribute to the responses, 
you can set them at a value that minimizes or maximizes your responses. Although the impact is minimal, might as 
well help yourself as much as possible. Now you are ready for RSEs!!! Note, to avoid having to run a DoE for EACH 
metric, try to come up with a set of common variables that capture ALL metrics. 


Creating Response Surface Equations 

Once you have down-selected your variables to something more manageable, you are ready to create your 
RSE for your metrics or responses. These are calculated using the following equation: 

R=b o+ 1 Z b i k i + 2AA + X M 

i=l i=l i=l j=i + 1 

where bi are regression coefficients for linear terms, bn are coefficients for pure quadratic terms, by are coefficients for 
cross-product terms (second order interactions), and x i? xj are the design variables and XiXj denotes interactions between 
two design variables. 

You will go through most of the same steps as you did with the Screening test. The only differences will be with the 
type of DoE and the method for analyzing the data. So, let me describe those aspects and you can refer to the previous 
section for the other stuff. 

Let’s say that we identified 7 variables that contribute to the responses of interest. Go back to JMP and go to 
the JMP Starter and click the DOE tab. Now click the Response Surface Design tab. 


JMP - JMP Starter 


File Edit Tables Rows Cols DOE Analyze Graph Tools View Window Help 

[P- ? ft El «$♦ ± P P + j| | (Ho Data Tables) j 



Ready | [~~[nUM B~ 

The following window will come up. As you did when you were creating the DoE for the screening test, add 
the number of variables and input the real names and values if you like. Say we have 7 variables, since the window 
already provides 2 variables, I just need to add 5. Then press Continue. 
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The following will come up. You will then get a bunch of DoE options as shown below. Typically, more runs 
are better since you have more data for the regression. So, let’s pick the Central Composite Design (CCD) with 144 
cases and 2 center points. Then hit Continue. 



The window below will then open up. Note, the number of center points is there for when you are doing 
experiments that have noise, such as variations in a control environment. Since we are running computer simulations, 
the experiment is 100% repeatable and we will only need 1 center point. So, in the cell where the number of center 
points is defined as 2, change that to a 1. Also, if the “On Face” option is not selected for the design you chose, please 
select it. The reason for this is beyond the scope of this tutorial, but trust me, just select it. Again, change the Run 
Order to “Keep the Same” option as you did before. Then hit Make Table. 
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Ready NUM 


Your DoE table will open up. As shown below. Now, go through the same procedure that you did for the 
screening test. 

1 . copying your table to Excel, 

2. convert to real numbers, 

3. saving as doe. table, 

4. ftp to your UNIX account, 

5. modify your shell scripts for the new number of cases and variable names, 

6. switch out variables, and 

7. parse your data. 

Bring the results back to Excel and import over to JMP. Now the difference is how you analyze the data. 
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The example that I have shown you thus far was for an HSCT aircraft concept. When I performed the 
screening test, I discovered that 16 variables were required to define the RSE. Within JMP, only an 8 variable RSE can 
be created. However, there are custom designs created by Dr. Oliver Bandte that can handle up to 16 variables. These 
designs are face-centered CCDs with a Resolution IV fractional factorial design. These custom DoEs allow for 
estimates of all main effects as well as all interactions between main effects. This is called a Resolution IV DoE. Then, 
the fractional factorial designs were merged with a center point in the hyper-cube and a set of face-centered axial 
points to form the higher than 8 variable CCDs. Regardless, the process to fit the data is the same and I will show you 
the one for the 16 variable. So, we open up the 16 variable design (which required 289 runs) and insert more columns 
for our 8 responses and then paste the data from our analysis code. 
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Now, let’s create the RSEs! Consider the example JMP file below. There are 8 responses and 16 design 
variables. Again, go to Analyze and Fit Model. Highlight your variables and instead of clicking the Add button as you 
did for the screening test, you want to click Macros ► . A drop menu will appear. Select the Response Surface option. 
The white area under the “Construct Model Effects” will fill out with the coefficients for a second order RSE that JMP 
will be solving for in the regression. Then, select your responses and put them in the Y area again. DON’T FORGET 
TO UNSELECT THE CENTER POLYNOMIALS OPTION UNDER MODEL SPECIFICATION! 




File Edit Tables Rpws Cols DOE Analyze Graph Tools View Window Help 
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One thing before you proceed: in the “Construct Model Effects”, you have your “to be determined” 
coefficients. Highlight all of the main effect variables that have the symbol “&RS” by them. Then click the Attributes 
drop menu and unselect the “Response Surface Effect”. The reason for this is that JMP only likes to have 8 variables 
and just gives you a warning when you fit the model. Doing what you just did suppresses the annoying little error. This 
does not affect the fit of the model at all. Once you have done this then hit Run Model. 



Oh, this is the little error window you would get if you didn’t de-select the Attributes. 


JMP - Error Alert 



Limit of 8 response -surface terms for curvature 
analysis. 


E 
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When JMP is finished fitting the model, similar windows will pop up as they did in the screening test. I want 
you to note some differences here. Look at the error bars (the dotted red lines) around the perfect model fit and 
compare those to the ones you saw in the screening test earlier. Note the HUGE spread. One would think that the 
original linear fit was better, but look at the 4 cases that are on the bottom. These cases actually did not converge in the 
analysis program. The particular combination of variable settings inhibited the program from converging. Thus, the 
values of “0” for each of the responses will mess up the model. There are a couple of solutions. First, go back to the 
DoE table and determine which variables might be affecting the failure. You do this by looking at the rows before and 
after the failed case and determine which combination of the variable settings is causing the program to fail. Then, you 
can modify the ranges and re-execute another DoE if there are too many failed cases. Another option is at your 
disposal, ONLY IF YOU HAVE A FEW FAILED CASES. Maybe a good rule of thumb is that you can exclude only 
2-3% of the total cases you executed before you should modify the ranges or the DoE that you ran. For 289 cases 
considered herein, I am going to “Exclude” these 4 failed cases from the model and allow a regression on the 
remaining 285 cases. 



If you look at the Residual Plot below you can see that there is a clumping of responses. Also, the Y-axis 
scale is one-third the size of the X-axis. This implies very large residuals and a poor fit. The Y-axis should be lass then 
one tenth of the X-axis for residuals to be considered reasonable. It is important to examine the scale as well as the 
shape. 
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To do this, put your mouse over one of the four dots in the “Actual by Predicted Plot” above and left mouse 
click one of them. You will see that the little dot gets bigger. Then hold down the shift key and click another, and 
another and another. Now, right mouse click and the window below will pop up. Select the Row Colors option and 
then pick one of the colors, say red. You will see that the 4 dots that you selected are now red. 



Go back to the DoE table window and scroll down until you find the highlighted rows that have little red dots 
beside them as shown below. 
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We want to exclude these 4 cases from the analysis, so, go under Rows and select the “Exclude/Unexclude” 

option. 



JMP will add 0 by each row. This means that these rows will be excluded from the model. You will lose 
some degrees of freedom, but if you only eliminate a few cases, you should be ok. Please look at any DoE or Response 
Surface book or the JMP Help menu for more information about degrees of freedom. Now go through the same 
process as above regarding fitting the model. 
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Now when we get our new window that has the “Actual by Predicted”, look at the significant difference in the 
error bars. They are much tighter! For a good “Actual by Predicted”, you want to have each of the dots (or cases) to be 
as close to the diagonal line as possible. The diagonal represents the perfect fit. AS the dots (or cases) move away from 
the diagonal, the error in the prediction increases. The red dashed lines around the diagonal represent the 95% 
confidence intervals of the prediction and the blue horizontal dashed line represents the mean of the response. If by 
chance the blue dashed line falls inside the red dashed lines, you have a VERY bad fit. 



For another check of you model, go to the little red T by the “Response TOGW” drop menu and click on 
Row Diagnostics and then select “Plot Residual by Predicted” option as shown below. 
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Then scroll down until you see the Residual plot. This plot is also called a “scatter plot”. The residual is the 
error in the fitted model and is the difference between the actual value of each observation and the value predicted by 
the fitted model. You typically want a nice random shotgun scatter of your error with a very small magnitude on the 
vertical scale with respect to the predicted values. The example below isn’t too bad. There are only a few points at the 
top that stand out as having high error. For only a few cases like this, you could simply exclude those from your model 
and refit. However, you should look at the particular cases and try to determine if there is a pattern of variable 
combinations that are inducing an error. If so, you might want to investigate your analysis code to determine if there is 
a modeling problem. 

Also, if there is a pattern to the residual (i.e., looks like a smiley face or a sine wave), a couple of things may 
be going on: 1) you may need a transformation of your dependent variables to get a random gunshot, or 2) you may 
need higher order effects of your model. 
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There are two ways to select the dots on the Residual plot. The first is to simply click on each dot 
individually. Another is to go up on the top menu and select the little lasso icon. Then draw a circle around the cases 
that are outliers by holding down the left mouse button and drawing the circle to “lasso the cases”! Then you can 
highlight them as you did before and identify the cases from the DoE table. For this example, excluding the 4 cases 
above helped the fit sufficiently that we can proceed. 
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There is an assumption that a second order model will fit the data and that the higher order terms clump into 
an error term with normal distribution. The foundation for this is that you are fitting a model based on a Taylor Series 
expansion. For a second order model, you assume that all higher order effects are negligible and can be lumped into an 
error term. For this assumption to be valid, that error term usually needs to be a standard normal distribution with a 
mean of 0 and a standard deviation of 1 . Checking this error distribution is a good way of determining if you have a 
good fit with the RSE. If the distribution is not normal, it implies that the model you fit is not good, you may have bad 
cases or you may need a transformation of inclusion of higher order effects. To find the percent error and the error 
dsitribution, you must save the predicted formulas. Under the little red T by Response TOGW, select Save Columns, 
then Prediction Formula. This will add a row to your DoE table called “Pred Formula TOGW”. 



Repeat this for every response so your data table looks like this: 


EZE 


y File Edit Tables Rows Cols DOE Analyze Graph Tools View Window Help 

I i □ a? ft# H @ I 


^injxj 

^jsjxj 


JIT? 


i ❖ i ± p p + 


J | HSCT _pred_f ormulas 


<> 

Acq $ 

RDT&E 

5/RPM 

Pred Formula TOGW 

Pred Formula TOFL 

Pred Formula Vapp 

Pred Formula FlyOver 

Pred Formula Sideline 

Pred Formula Acq $ 

Pred Formula RDT&E 

P 

Pred Formula 5/RPM | 


226.726 

16533.367 

0.13428 

1037671.55 

17848.3842 

177.805498 

116.969206 

112.660097 

226.311096 

16515.631 

0.13393736 

2 

218.469 

15766.52 

0.12951 

996543.756 

16140.4218 

174.255458 

115.303305 

112.041214 

219.074976 

15811.5169 

0.13004075 

3 

223.966 

16516.909 

0.12863 

980010.778 

14356.3112 

172.931356 

111.212599 

111.845598 

223.204892 

16454.9819 

0.12767393 

4 

218.165 

15703.406 

0.13121 

998637.167 

15835.6667 

174.616299 

114.31219 

112.012317 

217.855532 

15681 .6419 

0.13084278 

5 

222.026 

15991.163 

0.1346 

1037256.73 

16287.8524 

177.760896 

115.520774 

112.551681 

221 .742929 

15970.109 

0.13468993 

6 

210.871 

15398.187 

0.1181 

872775.002 

13767.5301 

163.457751 

112.624649 

110.391944 

209.756071 

15326.2343 

0.11678795 

7 

220.68 

16032.323 

0.13042 

996120.039 

13923.6607 

174.206915 

112.053931 

112.074335 

220.622879 

16032.9838 

0.12998688 

8 

216.374 

15745.416 

0.12431 

950160.77 

16037.4095 

170.201757 

114.379978 

111.454513 

217.124659 

15802.7563 

0.12489748 

9 

232.81 

16920.571 

0.14261 

1125693.07 

17363.0515 

185.285491 

115.511855 

113.859843 

233.718743 

16979.9653 

0.14363062 

10 

211.067 

15302.946 

0.12084 

903644.617 

14443.7741 

165.95008 

111.839167 

110.773405 

210.821423 

15280.4327 

0.12055319 

11 

222.31 

16198.352 

0.13083 

999919.235 

14710.0505 

174.882619 

114.438146 

111.980684 

221.970157 

16175.6309 

0.13052394 

12 

211.964 

15293.313 

0.12363 

937274.321 

15794.1753 

169.03895 

115.905913 

111.41708 

212.154581 

15306.432 

0.12411463 

13 

212.446 

15407.237 

0.12276 

929816.468 

14557.6903 

168.464801 

111.674216 

111.08941 

212.758018 

15430.4581 

0.12307785 

14 

213.202 

15527.064 

0.12188 

918613.321 

13786.9545 

167.498658 

111.883373 

110.924968 

213.3424 

15536.7267 

0.12187023 

15 

210.261 

15272.899 

0.11885 

895149.183 

13701.279 

165.213986 

110.806391 

110.647055 

210.549671 

15290.3372 

0.11933241 

16 

219.112 

15868.962 

0.129 

994384.95 

16330.1987 

174.165089 

115.592358 

112.031581 

219.632426 

15896.5659 

0.12967002 

17 

214.994 

15538.793 

0.12614 

950968.174 

14986.4836 

170.574243 

112.480586 

111.33834 

214.585695 

15510.4715 

0.12565139 

18 

217.449 

15784.445 

0.12703 

972615.896 

14796.8032 

172.322142 

112.665542 

111.667433 

217.846177 

15813.1172 

0.12749377 

19 

213.388 

15477.673 

0.12308 

930012.549 

13745.8847 

168.477755 

111.354843 

111.113585 

213.347586 

15460.9147 

0.12317751 ▼ | 


r HSCT_pred_f ormulas 


Columns (41/0) 


© Pattern 
© Wing_Area 


© OPR 
© CLdesign 


All Rows 

Selected 

Excluded 

Hidden 

Labelled 
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Now you want to find the percent error between the actual and the predicted responses. Add columns with 
Cols, Add Multiple Columns, and add eight columns after the last column. Rename the first column ‘error TOGW’. 
Each column will find the percent error for one response, so name them appropriately. Right click on the first of the 
columns and select “formula”. This window will pop up: 



Uj jJj 

Enter the formula for percent error using the column names: 

Percent error = (Predicted Value - Actual Value) / Actual Value *100 
NOTE: JMP does not like number entered first into the formula. If you are multiplying by a constant, do so at 
the end of the formula. 

After the percent error for each response has been calculated using the entered formula, you want to see if the 


distribution is normal. Go to Analyze, Distribution. 


J File Edit Tables Rows Cols DOE Analyze Graph Tools View Window Help 


^jejxj 

.jejxj 



3 Distribution 


** Fit Y by X 
Ffr Matched Pairs 
g Fit Model 
£_ Nonlinear Fit 
Multivariate 
** Cluster 

^ Survival ► 

JR Time <?Brific 

11 |T ? # H * 

f) * P P 

| HSCT_pred_f ormulas 

3 ! 

w HSCT _pred_1ormulas | 

i . ; 

i 

le 

Pred Formula Acq J 

Pred Formula RDT&E 

Pred Formula $/RPM BBSM error TOFL 

error Vapp 

error 

Flyover 

error | 
Sideline 

error Acq $ 

error 

RDT&E 

error 

$!RPM 



j 

97 

226.311096 

16515.631 

0.13393736 

-0.31395 

-1.01279 

-0.22138 

-0.24 

0.000071 

-0.183 

-0.10727 

-0.25517 


w Columns (41 fl) 

1 

14 

219.074976 

15811.5169 

0.13004075 

0.484751 

0.720261 

0.204403 

-0.00867 

0.000874 

0.277374 

0.285395 

0.409811 



paa- 


98 

223.204892 

16454.9819 

0.12767393 

-0.99237 

-0.1925 

-0.4998 

-0.17445 

-0.00145 

-0.33983 

-0.37493 

-0.74327 


IE) Pattern 
© Wing_Area 
© T/W 
© TIT 
© FPR 
© OPR 
© CLdesign 
© X2 
© X3 
© X4 
© X5 
© X6 
© Y2 
© t/c_root 
© t/c_tip 
© HT_ref 
© VT_re< 

© TOGW 
© TOFL 
© Vapp 
© Flyover 
© Sideline 
© Acq $ 

© RDT&E 
© J/RPM 

© Pred Formula TOGW E 
© Pred Formula TOFL B 
© Pred Formula Vapp l±l 
© Pred Formula FlyOver 
© Pred Formula Sideline 
© Pred Formula Acq $ 1+ 
© Pred Formula RDT&E • 
© Pred Formula $/RPM 4 


4 

219 

112.012317 

217.855532 

15681.6419 

0.13084278 

-0.25758 

-0.8784 

-0.10509 

-0.0888 

-0.00032 

-0.14185 

-0.13859 

-0.27987 


5 

1774 

112.551681 

221 .742929 

15970.109 

0.13468993 

-0.13088 

0.660357 

-0.13433 

-0.11819 

-0.00013 

-0.12749 

-0.13166 

0.066812 


6 

649 

110.391944 

209.756071 

15326.2343 

0.11678795 

-1 .37691 

-0.48048 

-0.51263 

-0.06303 

-0.00183 

-0.52873 

-0.46728 

-1.11096 


7 

1931 

112.074335 

220.622879 

16032.9838 

0.12998688 

-0.12214 

0.771952 

-0.11071 

-0.07562 

0.000055 

-0.02588 

0.004122 

-0.3321 


8 

1978 

111.454513 

217.124659 

15802.7563 

0.12489748 

0.549784 

-1 .3932 

0.236606 

-0.07305 

0.000952 

0.346926 

0.364171 

0.472589 


9 

855 

113.859843 

233.718743 

16979.9653 

0.14363062 

0.800595 

0.370261 

0.371339 

0.033701 

0.001119 

0.390337 

0.351018 

0.715675 


10 

1167 

110.773405 

210.821423 

15280.4327 

0.12055319 

-0.23959 

0.991288 

-0.15037 

-0.06558 

-0.0002 

-0.11635 

-0.14712 

-0.23735 


11 

146 

111.980684 

221.970157 

16175.6309 

0.13052394 

-0.30121 

-1.01574 

-0.00994 

0.251188 

-0.00082 

-0.15287 

-0.14027 

-0.23393 


12 

1913 

111.41708 

212.154581 

15306.432 

0.12411463 

0.381555 

0.178709 

0.141558 

0.053438 

0.000429 

0.089912 

0.085783 

0.391997 


13 

216 

111.08941 

212.758018 

15430.4581 

0.12307785 

0.389444 

-0.57581 

0.217014 

0.239351 

0.000255 

0.146869 

0.150716 

0.258919 


14 

373 

110.924968 

213.3424 

15536.7267 

0.12187023 

0.081387 

-1 .53582 

0.118743 

0.178842 

-0.00021 

0.065853 

0.062232 

-0.00802 


15 

1391 

110.647055 

210.549671 

15290.3372 

0.11933241 

0.398959 

0.097012 

0.19041 

0.113715 

0.000255 

0.137292 

0.114178 

0.405898 


16 

1358 

112.031581 

219.632426 

15896.5659 

0.12967002 

0.709805 

3.25113 

0.325512 

0.066655 

0.001178 

0.237516 

0.173949 

0.519394 


17 

1586 

111.33834 

214.585695 

15510.4715 

0.12565139 

-0.46672 

-0.29616 

-0.07367 

0.268116 

-0.00099 

-0.18991 

-0.18226 

-0.38735 


18 

1542 

111.667433 

217.846177 

15813.1172 

0.12749377 

0.533513 

0.610616 

0.303924 

0.137711 

0.000534 

0.182653 

0.181649 

0.365088 


19 

843 

111.113585 

213.347586 

15460.9147 

0.12317751 

-0.13185 

-1.39251 

-0.0132 

0.034235 

-0.00023 

-0.01894 

-0.10827 

0.079226 


20 

438 

112.352735 

221 .004456 

15962.5251 

0.13229871 

0.030554 

1 .558591 

0.070251 

0.034507 

0.000229 

-0.05949 

-0.09816 

0.036834 


21 

666 

113.263796 

228.817356 

16584.4006 

0.13881348 

0.222216 

-0.85671 

0.077075 

-0.20089 

0.000436 

0.09946 

0.146186 

0.139579 


22 

1293 

110.478342 

206.84984 

14967.8805 

0.1176155 

0.958129 

2.20555 

0.424624 

0.091768 

0.001425 

0.316123 

0.237325 

0.749957 


23 

1814 

111.684334 

218.562454 

15895.7305 

0.12766478 

0.2305 

-0.44045 

0.208013 

0.103779 

0.00008 

0.119768 

0.107172 

0.192105 


24 

078 

111.141091 

208.993823 

15048.2235 

0.12158976 

0.825575 

2.113332 

0.357925 

0.095213 

0.000987 

0.295532 

0.27545 

0.620455 


25 

1105 

112.855951 

223.41499 

16061.4038 

0.13738027 

0.283034 

0.901491 

0.12989 

0.067387 

0.000224 

0.164983 

0.130088 

0.219047 


26 

1046 

111.407126 

214.985711 

15661.9351 

0.12400995 

0.353057 

-1.21439 

0.004917 

-0.30403 

0.001034 

0.039419 

0.126255 

0.331674 


27 

756 

112.290903 

221 .445202 

16004.7964 

0.13246191 

-0.5878 

0.605779 

-0.30455 

0.006267 

-0.0011 

-0.30874 

-0.30579 

-0.29213 


28 

1908 

112.17459 

220.809796 

15978.0096 

0.13121807 

0.095855 

-1 .50459 

-0.07586 

-0.24872 

0.000439 

0.170025 

0.215337 

0.006152 


29 

1562 

111.41109 

217.624789 

15847.1149 

0.12586844 

-0.5333 

0.760706 

-0.11547 

0.274159 

-0.00103 

-0.22932 

-0.24262 

-0.51499 


30 

079 

111.507606 

215.011951 

15525.8464 

0.12652543 

-0.32927 

-1.33011 

-0.21006 

-0.13488 

-0.00024 

-0.05116 

0.096276 

-0.31088 


31 

439 

110.919924 

214.751776 

15769.2983 

0.12075203 

-0.35263 

0.481099 

-0.1905 

0.014861 

-0.00054 

-0.11685 

-0.17031 

-0.35317 


32 

201 

111.952927 

216.750058 

15577.5135 

0.13050964 

-0.86832 

-1.91637 

-0.5151 

-0.25854 

-0.00099 

-0.32601 

-0.31951 

-0.67759 


33 

1195 

113.927743 

235.559229 

16949.7357 

0.14973342 

0.908056 

0.306256 

0.429657 

-0.02811 

0.001758 

0.408451 

0.456427 

0.783077 


© error TOFL B 
© error Vapp B 
© error Flyover B 
© error Sideline B 
© error Acq $ l±l 
© error RDT&E B 

34 

1019 

111.26559 

223.961272 

16324.7464 

0.13181903 

0.134691 

-0.1944 

0.032619 

-0.00951 

-0.00005 

0.062224 

0.065805 

-0.00073 


35 

!568~ 

111.920861 

223.810608 

16138.8825 

0.13682318 

0.893407 

1.25027 

0.399914 

0.084716 

0.001325 

0.358101 

0.300908 

0.731197 


36 

014 

111.792911 

224.771489 

16280.6143 

0.13554958 

-0.2645 

1.476519 

-0.11891 

0.100419 

-0.00053 

-0.106 

-0.06821 

-0.38246 


37 

1232 

111.56132 

225.281392 

16411.7122 

0.13380477 

-0.02617 

-0.17425 

0.010316 

0.122499 

-0.0003 

-0.01625 

0.058786 

0.040951 


— 

38 

1996 

110.469818 

213.979778 

15394.5206 

0.12688071 

-0.68804 

-0.43732 

-0.25441 

0.231577 

-0.00125 

-0.40318 

-0.53211 

-0.4623 


39 

1587 

111.387872 

225.115122 

16468.8548 

0.13226843 

0.113406 

-0.54602 

0.001859 

-0.15687 

0.00043 

0.068956 

0.121485 

0.036628 


<r Rows 

40 

1305 

112.012775 

223.396145 

15995.3411 

0.1376423 

-0.92421 

-1 .52855 

-0.34622 

0.148453 

-0.00185 

-0.43759 

-0.44658 

-0.6408 


All Rows 289 

Selected 1 

Excluded 4 

Hidden 0 

Labelled 0 

42 

1108 

111 .830208 
112.151688 

224.080076 

227.768748 

1 61 58.7072 
16479.439 

0.1 3653392 
0.13821144 

-0.53896 

-0.42615 

-0.24706 

0.01 3881 
-0.09734 

-0.00071 

0.006728 

-0.20254 

-0.229 

-0.44555 


43 

193 

110.548121 

216.541331 

15726.2109 

0.12639931 

0.065218 

0.483487 

0.045465 

0.009067 

0.000321 

0.0214 

-0.00205 

0.03111 


jd 

1904 

112.126256 

225.284403 

16208.415 

0.13852169 

0.293955 

-1 .32828 

0.154635 

-0.02243 

0.000377 

0.177158 

0.142932 

0.312616 

III 




M. Kirby 


Page 38 




TIES for Dummies 


Place the error columns into the Y, Columns box. 


The distribution of values in each column 


-=id-x| 


-Select Columns - 
O Pred Formula TOFL 
© Pred Formula Vapp 
© Pred Formula FlyOvi 
© Pred Formula Sidelir 
© Pred Formula Acq $ 
© Pred Formula RDT&E 
© Pred FormulaJfRPM 

©i 



—Cast Selected Columns into Roles 


1 Y, Columns i| 

© error TOGW 



© error TOFL 
© error Vapp 
© error Flyover 

- 

* 

Weight | 

| optional Numeric 

Freq j 

I optional Numeric 


By 

optional 






The distribution for the error will pop up. This gives you a distribution, a box plot, quantiles, and moments 
about each column. Here you can check for normal distribution with a mean of 0 and a standard deviation of 1 under 
the Moments drop menu at the bottom. 


JMP - [HSCT_pred_formulas- Distribution] 


[£] File Edit Tables Rows Cols DOE Analyze Graph Tools View Window Help 

j|l D gSH# Jfc 3s e gj I 


^JSjxj 




HSCT _pred_formulas 




w w Distributions 


▼ - error TOGW 


H error TOFL 



:k 


¥ 


ir Vapp 


'T~ 


r 


ia 


r error Flyover 



▼ error Sideline 



ir Acq $ 






|S 


▼ Quantiles 

| 

^ Quantiles 

■ 

Quantiles 

■ 

▼ Quantiles 

■ 

▼ Quantiles 

■ 

* Quantiles 

u 

▼ Qua 

100.0% maximum 

1.849 

100.0% maximum 

3.380 

100.0% maximum 

0.847 

100.0% maximum 

0.7386 

100.0% maximum 

0.3382 

100.0% maximum 

0.851 

100.0-5 

99.5% 


1.826 

99.5% 


3.376 

99.5% 


0.813 

99.5% 


0.5830 

99.5% 


0.3280 

99.5% 


0.795 

99.5% 

97.5% 


1.312 

97.5% 


2.724 

97.5% 


0.636 

97.5% 


0.3099 

97.5% 


0.2379 

97.5% 


0.544 

97.5% 

90.0% 


0.888 

90.0% 


1.784 

90.0% 


0.427 

90.0% 


0.1792 

90.0% 


0.1341 

90.0% 


0.354 

90.0% 

75.0% 

quartile 

0.404 

75.0% 

quartile 

0.899 

75.0% 

quartile 

0.198 

75.0% 

quartile 

0.0904 

75.0% 

quartile 

0.0736 

75.0% 

quartile 

0.167 
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If the distributions are not normal you have three options: exclude cases, transform the responses with 
logarithmic or exponential functions, or include higher order effects. For this tutorial, we will focus on excluding 
cases. To select cases to exclude, you can click on the distributions or the box plots. Selecting a bar of the distribution 
selects all the cases within that bar. If you only want to select some of the cases within a bar, use control and the left 
mouse button to scroll the axis or zoom in with the magnifier in the tool bar. An interesting aspect of these 
distributions is that selecting a case in one selects it in all of them so you can see where the cases fall throughout all the 
distributions. 


| JMP - [HSCT _pred_formulas- Distribution] 



You want to exclude cases that have large percent errors to improve the fit of your RSEs. After selecting the 
cases you wish to exclude, it is a good idea to check if there is a pattern to those cases. In the DoE table, select the 
columns you want to compare by highlighting the name row. Go back to the distributions and double click on any of 
the selected points. This will bring up a table containing the information for those points. This is an easy way to find 
patterns and see if anything specific is causing the large error. 
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With the cases you want to exclude selected, go back to the data table and try excluding these. Rerun the 
analysis. You have to be careful how many cases are excluded. One way to judge if you have excluded too many cases 
is to look at the Correlation of Estimates. 



The Correlation of Estimates is a symmetric matrix that shows the mathematical correlation between the coefficient 
estimates. You should check that these numbers, except for the “l”s that make up the diagonal, are below ±0.15. If any 
number in this matrix exceeds ±0.25, you will not be able to differentiate between the influences of those coefficient 
estimates due to the prediction being correlated. What this implies is that if you are trying to estimate the coefficient 
for one term in your RSE and it has a high correlation with another coefficient, you will not be able to distinguish 
which of those coefficients is actually contributing to the response. This means you have excluded too many cases 
from your DoE and you cannot estimate your responses with your current data set and DoE. Check this matrix for each 
response when you exclude more than 5% of your original DoE cases. If your correlations start getting high, you need 
to modify your DoE by changing ranges, transforming your data, and/or check your analysis tool to see if you set up 
your model correctly. 
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Ready 



Looking at your actual versus predicted plot and the residuals plot, the fit of these two plots should reflect a 


better fit through the remaining cases. 

The most accurate means of determining how well your RSEs model your analysis code is to run a set of 
random cases. This is especially important if your responses are highly quadratic and there are very strong interactions 
amongst variables. To establish the interactions between variables, go to the red arrow beside “Response TOGW” and 


select the Factor Profiling and then the Interaction Plots. Scroll down in the window. 
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The plot below will come up. To identify if there are strong interactions between variables, look at one of the 
boxes. For example, consider the interaction between Wing_area and T/W. If these two variables had a small 
interaction between each other, the two lines that you see would be parallel. However, when interactions exist, the 
lines will have different slopes. Very strong interactions can be identified when one line crosses the other. What 
happens within the RSE for cases like this is that one effect can be masked by the other and when you evaluate your 
RSEs at values other than the cases used to create the RSEs, you may have significant prediction errors. However, if 
the interactions between variables are small and your responses are not extremely quadratic, your RSEs should behave 
well at off desig n po ints. 

I E* 




— jf 


1 










This is the interaction 
between WingArea 
and T/W. 


However, if you do have strong interactions and highly quadratic responses, you need to run a set of random 
cases. Especially if you don’t know a priori the behavior of a response. The suggested amount of random cases is an 
equivalent amount to the number of cases you ran for the original DoE. You can do this by randomly picking values 
between “-1” and “1” and creating a new DoE table. You can do this in Excel in your convert spreadsheet using the 
formula =2*rand( )-l. This will give you a value between -1 and 1. You will want to paste special these numbers as 
values into the convert sheet because they have a tendency to change with any change in the spreadsheet. Re-execute 
your analysis tool for those random cases to get your “Actual Response Values”. Bring the responses into the DoE 
with the newest prediction formulas calculated after excluding cases and save as a new file. Bring in the random 
values, which will automatically update the values in the predicted formulas and the error columns. Now you can look 
at the error distribution to see how the RSE performs at off design points, again look for a normal distribution. An 
acceptable level of error is ±5%. If your error is higher than this, you should re-examine your ranges and the DoE that 
you used. For a given design problem, this effort should be done at least once to get a feel for how well a 2 nd order 
RSE can capture a given response. This is a valuable one-time investment. 
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For one last test to see how well the RSE is predicting your analysis code, click the Summary of Fit button 
and observe the “Rsquare” value. Rsquare is a number that indicates the accuracy of your predicted graph. A value of 
one indicates that the relationship is perfect, while zero indicates no relationship whatsoever. As you can see it has a 
value of 0.993631. This means that the second order DoE model that you chose explains over 99% of the variation in 
the data. With this check for the goodness of the fit of your model, you typically want a value greater than 90%. 
However, the R 2 value IS NOT THE ONLY CHECK FOR THE GOODNESS OF FIT! The R 2 tells you how well you 
are predicting the responses at the values prescribed by your DoE table. It DOES NOT tell you how accurate your 
responses will be at variable settings other than “-1”, “0”, or “1”. This value will be closer to one if you have excluded 
cases, because your RSE is doing a better job of modeling the variations of the data. 


^ JMP - IHSCT feasibility RSE: Fit Least Square si 



Another great feature of JMP is the ability to see contour plots. So, go under the Least Squares Fit drop 
menu again and select the Profilers option again, but now select the “Contour Profiler” option. 
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Scroll down in the window to see the “carpet” plot shown below. Here is where you can play with constraints 
that may be hurting you. You can change which variables are displayed in the contour by simply clicking in the boxes 
under “Horiz” and “Vert”. You can assign specific values to your responses by clicking in the boxes under “Contour” 
and adding values. You can put limits on the responses as either Hi (high) or Lo (low) limits. You can set the design 
variables to different values by moving the slide bars, etc. For presentation purposes, copy this in the same manner as 
you did before. An example of the contour plot is shown below. There are upper limits on 2 responses, and the design 
variables are all set at “0”. Note that the contour values were set to the limit values so that you can visualize which 
constraints are active. In this case, the TOGW and the TOFL are active constraints. Just play with this, it is VERY 
easy. Note, to expand the contour plot for easier viewing or presentation, put your cursor in the bottom right hand 
comer and then click and drag the corner of the box. You can also turn off the Surface Plots if you like under the drop 
menu for the Contour Profiler. You can zoom in and out on the contour plot by putting your mouse over one of the 
axes (either T/W or Wing Area), hold the “Ctrl” key down and then right mouse click and select the “Size/Scale” 
option and then either X or Y axis. If you select the Y-axis, you get the window down below. You can modify any 
range you like. 


4$ JMP - [HSCT_feasibility_RSE: Fit Least Squares] 
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One item of interest to you might be the following: What are the settings of the variables that optimize my 
responses? JMP can optimize your variables for you through a feature called the “Desirability Function”. Consider our 
8 responses, each of which we would like to minimize. We need to tell JMP this information. Go back to the DoE table 
and click on the first response column title cell, TOGW, and hold down the shift key and click the last response title 
cell ($/RPM). Note that all of the column headings are highlighted. Now, right mouse click and you will get the 
following menu and select the Column Info option. 



The window below will come up. 



M. Kirby 


Page 46 





TIES for Dummies 


Under TOGW, select the New Property button. And then select the “Response Limits” option. 



The following will come up. Click the Maximize button and change it to the “Minimize” option. This tells 
JMP that the response is to be minimized when we do the search for the most desirable settings. Repeat this for each 
response and then click the Apply button and then OK and go back and Fit Model again. 


<$► JMP - [Column Settings] 
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Go back to the “Least Squares Fit” window and go to the Prediction Profiler and click on the little red ▼ and 
select the “Desirability Functions” option. Notice that the “Confidence Intervals” are checked in the box below. The 
Confidence Intervals are the little error bars on the Prediction traces of the Profiler. The larger they are, the more error 



After you select the “Desirability Function” option, another column will appear in your Profiler and another 
row. On the right, this is the direction of the desired response; it is shown in minimization right now. If any of the 
responses were to be maximized, then the slope would be positive, rather than negative as shown. On the bottom, the 
influence of a variable on the desirability is shown. For example, increasing Wing Area increases the desirability, 
reducing T/W increases desirability, and so on. JMP will automatically find the optimal settings by selection the 
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JMP will then come back with the most desirable settings for the responses as shown below, if it in fact can 



Also, there is another way to change the desirabilities. Put your cursor in one of the desirability boxes on the 
right and hold down the “Alt” key and click. The window below will open up and you can change the options. 



Let’s summarize what we should have done up to this point. 

• Look at the “Actual versus Predicted Plots” to check the goodness of fit 

• Look at the “Residual Plots” 

• Check the error distribution 

• Exclude bad or high error cases 

• Run random cases if you have strong interactions and large quadratic behaviors 

• Check the R 2 value for EACH response 

• If an acceptable value is obtained, we can proceed, if not, then we need to check what went wrong 

• Modify ranges if needed and run another DoE 

• Refit data 

• Create Contour Plots and Prediction Profiler for presentation purposes 
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Let’s press onwards with investigating the design space. We now want the coefficients for all of the 
responses. The easiest way to do this is to go back to the Fit Model window and select the Manova option under 
“Personality” menu then select Run Model. 



The window below will come up. Select the little cross icon from the menu bar and then click anywhere under 
the Parameter Estimates bar and all of the area will be highlighted. You will have an Excel file called 
Lots _of_RSE_Eqn setups, which contains spreadsheets to calculate the RSEs for 5 to 29 variables. 
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Open Excel and the Lots of RSE Eqn setups file. Go to the sheet that has the appropriate number of 
variables. Right mouse click on the sheet tab name as shown below and select “Move or Copy. . .” 



The following window will come up. Under “To Book” select “(new book)” and click the “Create a copy” at 
the bottom. 



Your window should look like this. . . 
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Go back to the Lots of RSE Eqn setups sheet and close it. Save your new file as “Feasibility_Investigate”. 
Click Insert and select “Worksheet”. Go back to JMP and copy the highlighted area by going to Edit and select Copy. 
Go back to Excel and go to Edit and then Paste Special, and then paste as Unicode Text. Your RSE coefficients will 
then be pasted into different cells on the new worksheet. 
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For this example, switch to the “16_var” sheet. Note, this template was set up for only 5 responses (Y1 
through Y5). We have 8 responses so we need to add some more columns. To do so, click in column “I” header and go 
to Edit and select Copy. Then go to Insert and select Copied Cells. Excel adds an exact copy as you can see below. 
You need to copy the last column of responses (in this case Y5) and then insert that column. Make sure you check that 
the formulas were copied. Change the generic response names from Y1 and so on to your actual response names (i.e., 
TOGW). You should also put them in the row that contains the cell called “Parameters for:” to the right of all of the 


zeros. 
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Go back to the sheet where you pasted your coefficients from JMP and highlight cell A3 through the bottom 
of the coefficients, in this case that would be cell 1155. Copy and then switch back to the “16 Var” sheet. Make the 
active cell the one DIRECTLY under the cell with “Parameters for:” in it. In this case, it would be J5 and then paste 
your info as shown below. You will see that column E through L fills out with numbers besides “0”. Now, you need to 
change the input variables (XI through XI 6) to your actual variable names. In this case, XI corresponded to 


WingArea, X2 was T/W, and so on. Do this for cells A6 through A21 and then A25 through A40 and A46-A61. You 
should put in your baseline values (in dimensional form) in cells B6 through B21. Then add the ranges that you used in 



Once you have supplied all the info^now you are ready to run C rystal Ball. 
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Running Crystal Ball: A Monte Carlo Simulation 


This is probably the easiest and quickest part of the whole method. You should still have your Excel 
spreadsheet from above open. Crystal Ball is already linked to Excel. If you do not have the drop menus in Excel 
entitled Cell and Run, contact Dr. Kirby. Else, let’s continue. First you need to tell Crystal Ball which cells you want 
to vary. In this example, you will be doing your 7 design variables. So, highlight the 7 cells as shown below. Then, go 
to Cell and select Define Assumption. 
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You will get the below window. For the feasibility investigation (i.e., design variable variation), you want all 
of your variables to be defined by a uniform distribution. Click in the uniform region and then hit OK. 
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Your first variable will come up as shown in the next window. In this example, it is AR 
You need to modify the distribution range by clicking in the Min and Max cells and changing the 
you used in your DoE. 
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This is shown in the following window. Once you have done that, click OK. Repeat this for EVERY design 
variable. When you are done, the cells that you highlighted willj;hange colors. This means they are ready to go. 



Now you need to define your forecasts (i.e., what you are interested in tracking). In this case, it is the 
responses TOGW, TOFL, etc.. Highlight these 8 cells as shown below. 



Click the Cell menu and select the Define Forecast option. You will get the following window. This is telling 
Crystal Ball that these cells are what you want to keep track of. Make sure that the Forecast Window Size is selected 
as Small, and the window is displayed When Stopped. Then click OK. Another window will pop up for the next 
metric. Do the same thing for all and when you are done, the forecast cells will change colors. Also, if the correct name 
of your response is not listed beside the “Forecast Name:” you may enter it at this time. 
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Go to the Run menu and select the Run Preferences option. Click on the Trials button and make sure it 
looks like the one below. 



Then click on the Sampling button and make sure it looks like the one below. 



Then click on the Speed button and make it look like this one. 



Then click on the Options button and make it look like this one. 
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You are now ready to run Crystal Ball!!!!!!. Go to the Run menu and select Run. Excel will automatically 
minimize itself to run faster. When Excel/Crystal Ball is done running the 10,000 simulations, it will pop back up with 
the following. Click OK. 


Crystal Ball 



Maximum number of trials reached, 


\E\ 


OK 


Help | 


Once you click OK, the forecast windows will pop up as shown below. The windows are tiled and are the 
frequency distributions for your metrics as a function of the 7 design variables you had. You can sit and play with all 
of these windows. Look at the different options and the preferences. Crystal Ball is very straightforward. 
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Ready 


One of the more interesting features of Crystal Ball is the Overlay Option. Go to Run and select Open 
Overlay Chart and the window below will open. Click the button Add Forecast. . . . 


Edit Preferences View Run Help 
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Help 
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The window below will open. Now you can add different forecasts (responses) and look at them concurrently. 



Let’s look at the Flyover and the Sideline Noise. Click the Flyover and then click Choose, do the same for 
Sideline and select OK. The window below will come up and show you where the design space lies for the Sideline 
and Flyover noise level. Play with the “Preferences" and “Views” to get exactly the picture you want. You can also 
copy the Overlay chart by going to Edit and then Copy. 



What you really want from the Monte Carlo Simulation are the values of the metrics for different probability 
levels. So, go under Run and select Extract Data and the window below will pop up. Make sure yours looks like the 
one below. Click OK. Or you can extract other info that interests you. 
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Excel/Crystal Ball will open another worksheet that looks like the one below. These are your response values 
for different probability percentages. In this form, you can make plots in Excel and overlay any other info you want. 
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5 / Determine System Feasibility and_ Viability 

Once you have the results of the Monte Carlo Simulation, you want to compare your results to your targets. 
An example plot is shown below. The TOFL is the metric. The constraint is 11,000 ft as shown. The probability, or 
amount of the design space investigated, that will satisfy the constraint value is 31.5%. You need to make a plot like 
this for every metric that you are considering. Recall the CDF that you have represents all the possible geometric 
combinations as bound by the design variable ranges that you defined. Finally, tabulate your results and find out which 
constraint is hurting your feasibility and viability. 



Takeoff Field Length (ft) 
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Step 6: Specify Technology Alfernafives 

The objective of this step is three fold. One, identify potential technologies that may improve technical 
feasibility and economic viability of your vehicle. Second, establish physical compatibility rules for the different 
technologies identified. Third, determine the expected impact, both improvements and degradations, to the system of 
interest. The impact of a technology can be qualitatively assessed with technology metric “k” factors. These “k” factors 
modify disciplinary technical metrics, such as specific fuel consumption, cruise drag, and/or component weights that 
result from some analysis or sizing tool. The modification is essentially a change in the technical metric, either 
enhancement or degradation. In effect, the “k” factors simulate the discontinuity in benefits and/or penalties associated 
with the addition of a new technology. Once you have identified the impacts, you need to match the “k” factors to the 
appropriate FLOPS/ALCCA variables and namelists. 

For step 6, the potential technologies are identified from the Morphological Matrix you created in Step 2. For 
the HSCT, eleven technologies and technology programs are considered. The technologies along with the primary 
purposes were identified through a literature search of potential sub-component. It is important to identify technology 
that will assist “show-stoppers”. In the HSCT example, the “show-stoppers” are noise and economic factors. Two 
technologies that assisted the sound metrics are being considered. The other technologies selected are an attempt to 
improve the economic metrics through secondary effects. The “Show-stoppers” have a large influence on what 
technologies you select to consider adding to your design. 


Technology Readiness Level 

Many of the identified technologies are in the development stage. It is important to identify how advanced 
they are and one way to do this is to use the technology readiness level. The technology readiness level (TRL) 
represents the amount of progress that has been made on a technology. On a scale of one to nine, the higher TRL a 
technology has the more tests and integration it has gone through. A higher TRL is also associated with less 
uncertainty concerning the final impact, so when evaluating the impact of the technologies on the vehicle, less 
uncertainty will need to be incorporated. The TRL of a technology affects the certainty of the outcome, and when 
evaluating technologies with lower TRLs on the vehicle there is a lower probability of having a feasible and viable 
design. 


Table 6: Technology Readiness Levels 


Description 

Level 

Qualifier or Development Hurdle 

Basic Research 

1 

Basic scientific/engineering principles observed and 
reported 

Feasibility 

Research 

2 

Technology concept, application, and potential benefits 
formulated (candidate system selected) 

Feasibility 

Research 

3 

Analytic and/or experimental proof-of-concept completed 
(proof of critical function or characteristic) 

Technology 

Development 

4 

System concept observed in laboratory environment 
(breadboard test) 

Technology 

Development 

5 

System concept tested and potential benefits substantiated in 
a controlled relevant environment 

System 

Development 

6 

Prototype of system concept is demonstrated in a relevant 
environment 

System 

Development 

7 

System prototype is tested and potential benefits 
substantiated more broadly in a relevant environment 

Operational 

Verification 

8 

Actual system constructed and demonstrated, and benefits 
substantiated in a relevant environment 

Operational 

Verification 

9 

Operational use of actual system tested, and benefits proven 
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Table 7: HSCT Technologies 


(Identifier) Technology 

■■an 

Primary Purpose 

(Tl) Composite Wing 

3 

Wing weight reduction 

(T2) Composite Fuselage 

3 

Fuselage weight reduction 

(T3) Circulation Control 

4 

Increased low speed performance 

(T4) Hybrid Laminar Flow Control 

3 

Cruise drag reduction 

(T5) Environmental Engines 

3 

Reduce noise, fuel burn, and emissions 

(T6) Advanced Flight Deck Systems 

4 

Synthetic vision removes fuselage nose droop weight penalty 

(T7) Advanced Propulsion Materials 

3 

High temp, materials, reduced engine weight, lower fuel bum 

(T8) Integrally Stiffened Aluminum Wing Structure 

4 

Wing weight and part complexity reduction 

(T9) Smart Wing Structures 

3 

Reduced flutter and wing weight 

(T10) Active Flow Control 

3 

Cmise drag reduction 

(Til) Active Acoustic Control 

3 

Noise suppression 


Compatibility Matrix 

It is not possible, however, to add all of these technologies onto a single design because some are 
incompatible. Technologies are incompatible when they compete for the same space on the aircraft or when one 
technology causes extreme degradation in the function or integrity of another. For example, technologies 1 and 8, the 
composite wing and the integrally stiffened aluminum wing structure, are incompatible because they are competing 
wing material technologies. All of the incompatibilities in the technology compatibility matrix (TCM) below result 
from this competition for the same design space or from extreme degradation effects. 

Now you need to provide a compatibility matrix for your technologies. This matrix formalizes which 
technologies are physically compatible and as a by-product, reduces the number of alternatives to evaluate. This is 
important if the number of technologies considered for application is large and the combinatorial problem is out of 
hand. A sample compatibility matrix is shown below for 11 technologies. A “1” represent compatible technologies 
while a “0” implies an incompatible combination. The TCM is a symmetric matrix so only half of it is filled in. A TCM 
is easily created by researching the possible technologies and seeing which ones compete for design space or severely 
degrade the intended function or integrity of the technologies. 
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Technology Impact Matrix 

Once the compatibility matrix is determined, the influence of infusing these technologies must be determined. 
This is difficult to evaluate directly, but it can be quantitatively evaluated through “k” factors. These “k” factors 
describe the potential system and sub-system level impacts of each technology. The impact must include benefits and 
degradations to the entire system. These “k” factors allow for the impact of the technologies to be evaluated using an 
M&S environment. 

The impact of each technology is determined through physics based modeling, literary research and 
questioning experts. This impact is based on the upper limit of the technology at full maturity and widespread 
application. The impact for each “k” factor defines a “k” vector for the technology. A technology may not have an 
influence in all of the “k” factors. The “k” vectors for each technology are combined into the Technology Impact 
Matrix (TIM). 

An example TIM is shown below for the HSCT example. Also note to the right is the max and min values that 
a given “k” factor could ever achieve if all technologies were “on”. The minimum and maximum values define the 
ranges you will use for the generation of your metric RSEs as a function of “k” factors, in lieu of design variables. 


Technology Impact Matrix (TIM) | 
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+2% 

+2% 

+2% 

+2% 


+2% 

-2% 

+2% 

+2% 

+1% 

-2% 

+17% 

RDT&E 

+4% 

+4% 

+2% 

+2% 

+4% 

+2% 

+4% 


+5% 

+5% 

+5% 

0% 

+39% 


Production costs 

+8% 

+8% 

+3% 

+5% 

+2% 

+1% 

+3% 

-3% 

-3% 

-3% 

-3% 

-12% 

+30% 
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Now you need to map these “k” factors to actual inputs to FLOPS/ALCCA (or your analysis code) and 
useable values. An example is shown below for the TIM provided above. Each of the “k” factors is mapped to a 
FLOPS/ALCCA input and namelist. The baseline values are established so as to determine how to deviate the input 
variable in accordance with a DoE. For example, the baseline engine weight is 9,238 lb. The minimum value is 
determined by the following: 9,238+9,238*(-0.1)=8314.2. The maximum is then 9,238+9,238*(0.46)=13487.48. 
Hence, the non-dimensional baseline value is -0.6428. You need to understand the dimensional minimum and 
maximum of your “k” factors if the non-dimensional impact is NOT symmetric about your baseline dimensional 
values. This is important for when you map the technologies to your RSEs. 








Dimensional impact 

Non-dimensional impact 



Variable 

Namelist 

Baseline 

value 

N on-Dimensional 
Baseline Value 

Min 

Max 

Min (%) 

Max (%) 



Wing Weight 

FRWI 

WTIN 

1 

0.666666667 

0.65 

1.07 

-35 

7 



Fuselage Weight 

FRFU 

WTIN 

1 

1 

0.60 

1.00 

-40 

0 



Engine Weight 

WENG 

WTIN 

9238 

-0.642857143 

8314.20 

13487.48 

-10 

46 

£ 


Electrical Weight 

WELEC 

WTIN 

1 

-1 

1.00 

1.22 

0 

22 

o 

o 


Avionics Weight 

WAVONC 

WTIN 

1 

-1 

1.00 

1.21 

0 

21 

03 

Ph 


Surface Controls Weight 

FRSC 

WTIN 

1 

-0.333333333 

0.95 

1.10 

-5 

10 

u 


Hydraulics Weight 

WHYD 

WTIN 

1 

0 

0.95 

1.05 

-5 

5 

o 

B 


Noise Suppression 

FSUPP 

n/a 

1 

1 

0.79 

1.00 

-21 

0 

<D 


Subsonic Drag 

FCDSUB 

MISSIN 

1 

1 

0.81 

1.00 

-19 

0 



Supersonic Drag 

FCDSUP 

MISSIN 

1 

1 

0.76 

1.00 

-24 

0 

B 


Subsonic Fuel Flow 

FFFSUB 

ENGDIN 

1 

0.333333333 

0.94 

1.03 

-6 

3 

o 

<D 


Supersonic Fuel Flow 

FFFSUP 

ENGDIN 

1 

0.714285714 

0.94 

1.01 

-6 

1 

H 


Maximum Lift Coefficient 

CLMAX 

AERIN 

1 

-1 

1.00 

1.15 

0 

15 



o&s 

AKOANDS 

IWGT 

0 

-0.789473684 

-0.02 

0.17 

-2 

17 



RDT&E 

AKRDTE 

IWGT 

0 

-1 

0.00 

0.39 

0 

39 



Production costs 

AKPRICE 

IWGT 

0 

-0.428571429 

-0.12 

0.30 

-12 

30 
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Step 7: Assess_ Technology Alternatives 

The technologies identified in Step 6 are now applied to the vehicle concept and evaluated. The evaluation 
provides data and information to the decision-maker whereby selection of the proper mix of technologies is performed. 
Yet, the search for the mix that will satisfy the customer requirements is dominated by the “curse of dimensionality”. 
Depending on the number of technologies (n) considered, the combinatorial problem can be enormous. If all 
combinations are physically compatible and assuming only an “on” or “off’ condition, then 2 n combinations would 
exist. In addition, the technology “k” factor vector that influences a vehicle is probabilistic and a cumulative 
distribution function (CDF) must be generated for each combination, further complicating the evaluation. If the 
computational expense of the analysis is acceptable, a full-factorial investigation could ensue. For the purpose of this 
tutorial, the evaluation will be completed deterministically, then probabilistically. 

For this tutorial, the computational expense is manageable due to the means by which the technology “k” 
vectors are modeled. Consider the TIM given before and a metamodel representation of a system response. If one were 
to bind each “k” factor element of the technical vector, a metamodel in the form of a second-order Response Surface 
Equation (RSE) could be generated for each of the system level response. Hence, the system response could be defined 
as a function of the “k” factors for a fixed geometry using the equation below, through a Design of Experiments. 

k k k — 1 k 

R = b 0 +Y, b i k i + + X XWi (!) 

i = 1 i = 1 i = 1 j=i + 1 

To understand the use of the RSE, consider a single technology (Tl). If an RSE was generated for three “k” 
factors: k 1? k 2 , and k 3 , it would take on the form: 

3 3 2 3 

^|71 = b 0 + ty kf + bn kj + bfj kj kj ) ^ 

i=l i - 1 i=\ j=i + 1 

R\t\ = b n + (b { k\ + b 2 k 2 + b 3 k 3 + b { x kf + b 22 k 2 + b 33 k 3 + b n k x k 2 + b u k { k 3 + b 23 k 2 k 3 ) !n 

The impact of Tl is to reduce kj by 10%, increase k 2 3%, and to have no impact on k 3 . Using these values in 
the RSE yields: 

R [t 1 =b 0 + Ztj (-1 0%) + (3%) + b x j (-1 0%) 2 + b 33 (3%) 2 + b u (— 10%)(3%) 

This procedure is repeated for all of the technologies and metrics that are considered. The coefficients (b 0 , bi, 
bn, etc...) are determined by the least squares analysis of the DoE 

To find the RSE, go through the same process that you did in the design space exploration. 

• generate a DoE or use one that was created if you have more than 8 variables, 

• copy the table to Excel, 

• save as “doe.table”, 

• take the “doe.table” to UNIX and run the appropriate shell scripts, 

• extract data, and 

• bring data back to JMP and create your RSEs just as you did with the design space 

When you go to the model fit of your RSEs, you will get the Technology Impact Forecasting (TIF) 
environment, as shown below. This environment allows you to see which discipline will help you most with respect to 
improving certain metrics. For example, you can see that “knoise” significantly reduces the SLN (sideline noise) and 
FON (flyover noise). In addition, “ksupersonicdrag” significantly reduces almost every metric. This is a good 
environment to see if you can even get anywhere near your response targets. For example, look at the minimum and 
maximum values of the response (on the left). These values represent some arbitrary combination of “k” factor 
settings. If the minimum or maximum does not contain the desired metric constraint value, then the technologies that 
you are considering will not help you. NO MATTER WHAT COMBINATION. You should identify technologies that 
are more aggressive. 

Another aspect of this Profiler is that you could reverse engineer the problem and determine what values of 
the “k” factors create a feasible configuration. This is the heart of the TIF method. Thus, once the “k” factor values are 
established, the decision-maker must identify specific technologies providing the predicted values. The reverse 
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approach was taken herein, such that specific technologies were identified for infusion, and the TIF environment was a 
fallout of this approach. 

Go through the same process of getting your RSE coefficients by analyzing with a Manova as you did 
previously. Again, take the RSE’s to the Lots _of RSE Eqn setups Excel file. Save the file as what ever you like. One 
suggestion though is to get rid of the extra sheets that you don’t need to save some memory. 




M. Kirby 


Page 66 


TIES for Dummies 


Now, you also need to create another JMP file that has a full-factorial DoE for your technologies. To do this, 
go back to the JMP Starter menu and select the Full Factorial Design button. 



And the following screen will pop up. To add the number of technologies that you have, just hit the 
Continuous button for as many technologies as you have. In our case, we have 11. So we hit the Continuous button 
1 1 times and then select Continue. 
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When you hit Continue the window will expand and look like the window below. Again change the “Run 
Order” to “Keep the Same”, but DO NOT add a center point and then hit Make Table. 



When you are done you will get the DoE below. As you can see, there are 2,048 rows which corresponds to 
2 n , where n=ll (the number of technologies). The factor is 2 since you are only going to consider an “off’ or “on” 
condition, i.e., “-1” or “+1”. Now you can change the variables from XI to T1 and X2 to T2 and so on. Add the 
appropriate number of response columns and save the file. 
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Deterministic Evaluation 

You will be given a spreadsheet entitled “CalcdeterministicFullfacttech”. This spreadsheet will take the 
above full factorial design and map the technology “k” factors to the metric RSEs generating deterministic results. The 
spreadsheet contains 5 sheets: “Full_Factorial”, “Results”, “kfactorRSE”, “Check Compatibility”, and “Compatible 
List” (this one is needed for TOPSIS in step 8) as shown below. The “Full_Factorial” sheet is where you copy your full 
factorial design created in JMP. As you can see, the technologies are listed on the left and on the right are the “k” 
factors. The purpose of this spreadsheet is to map a specific technology to the “k” factor values, then convert the 
dimensional values to the non-dimensional form, feed those numbers into the “k factor RSE” and calculate the RSEs 
and copy the results back to the results sheet. As you see below, this sheet maps the technology to the individual “k” 
factors. For example, some of the equations contained in the cells under the “k” factors are provided below, including 
the wing weight, engine weight, and hydraulics weight. The conventional value for these “k” factors, i.e. no 
technology added, is “1” and 9238 for the engine weight. Hence, if the composite wing is “on”, e.g., $B4=1, then add 
-0.2 to the wing weight value of 1, else, don’t add anything, “0”. And if the composite fuselage is “on”, e.g., $C4=1, 
then add “0” to the wing weight value of 1, else, don’t add anything, “0”, etc. Of note, consider the engine weight 
equation. The conventional technology weight is 9,238 lbs. Now, if a technology is “on”, then the 9,238 lbs will 
change by a certain percentage as shown in the equation below. It is important to check that the equations match the 
“k” factors in the TIM. This spreadsheet is set up for 11 technologies and 16 “k” factors. If you have less than 11 
technologies and 1 6 “k” factors, you need to modify the sheet. Also, if you have different technologies than the ones 
listed, you need to modify the spreadsheet. If you have no clue how to do this, just ask Dr. Kirby. 

Wing weight: 

=l+IF($B4=l,-0.2,0)+IF($C4=l,0,0)+IF($D4=l,0,0)+IF($E4=l,0.05,0)+IF($F4=l,0,0) 

+IF($G4=1,0,0)+IF($H4=1,0,0)+IF($I4=1, -0.1, 0)+IF($J4=l, -0.05, 0)+IF($K4=l, 0.02, 0)+IF($L4=l, 0,0) 

Engine weight: 

=( 1 +IF($B4= 1 ,0,0)+IF($C4= 1 ,0,0)+IF($D4= 1 ,0,0)+IF($E4= 1,0.01 ,0)+IF($F4= 1 ,0.4,0)+IF($G4= 1 ,0,0) 

+IF ($H4= 1 ,-0. 1 ,0)+IF ($14= 1 ,0,0)+IF ($ J4= 1 ,0,0)+IF ($K4= 1 ,0,0)+IF ($L4= 1 ,0.05 ,0)) *923 8 

Hydraulics weight: 

= 1 +IF ($B4= 1 ,0,0)+IF ($C4= 1 ,0,0)+IF ($D4= 1 ,-0.05 ,0)+IF ($E4= 1 ,0,0)+IF ($F4= 1 ,0,0) 

+IF($G4=1,0,0)+IF($H4=1,0,0)+IF($I4=1,0,0)+IF($J4=1,0.05,0)+IF($K4=1,0,0)+IF($L4=1,0,0) 
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Now look at the sheet called “Results”. In cell B4, the info from translating the impact of a mix of 
technologies should be copied into these cells. If you look to the right, starting in column “T”, you will see where the 
responses are. At present, the response columns are empty since you haven’t actually calculated the RSEs. How might 
you do that? Well, go to Tools and then select Macro and then select Macros. 



Then you will see the following window with the macro contained in this spreadsheet. It is entitled 
“runtechs”. Let’s take a look at what that macro does. Select “runtechs” if it is not highlighted and then select the 
Edit button. 



M. Kirby 


Page 70 





TIES for Dummies 


Then the following window will pop up, if you have the Visual Basic Editor. The macro is commented as to 
what it is doing. There are a few hitches here. The macro is set up for 1 1 technologies for a full factorial evaluation 
(2,048 cases) and for 16 “k” factors and 8 responses. This is important due to where the cells are being referenced. For 
example, the list of the full factorial cases starts in row 4 and column 2 (or B4) on the sheet “Results”. Since there are 
2,048 combination, the index “i” goes from 4 to 2,051. Next, the first occurrence of index “j” corresponds to the 
number of “k” factors and the second occurrence corresponds to the number of responses. Your response values must 
start in cell E3 on sheet “k factor RSE”. If they do not, you must modify the references in the macro. Also if you have 
more or less technologies, then you must change the index for “i”. Finally, if you have more or less “k” factors, then 
modify the value of the first occurrence of index “j” and if you have more or less responses, modify the second 
occurrence of index “j”. If you modify anything, then save the work and return to Excel. 


e 


Microsoft Visual Basic - Calc deterministic Full feet tech aIs 


File Edit View Insert Format Debug Run Tools Window Help 


atpvbaen.xls (ATPVBAEN.XLA) 
Bootstrp.xla (Bootstrp.xla) 
CB.xIa (CB.xIa) 

Decision.xla (Decision.xla) 
Double. xla (DOUBLE.XLA) 
funcres (FUNCRES.XLA) 

Matrix. xla (Matrix.xla) 
OptQuest.xIa (OPTQUEST.XLA) 
Tornado.xla (TORNADO. XLA) 
VBAProject (Calc_deterministic 
3 Microsoft Excel Objects 
j ® Sheet 1 (Full_Factorial) 
iff] Sheet2 (Results) 
gQ Sheet3 (k_factor_RSE) 
iff) Sheet4 (Compatible List) 
e ThisWorkbook 
3 Modules 

Module 1 


ss m - IS 


ed m\\E 


► ii 


lM ^ OS* W ^ | (?) Ln 16, Col 102 


Calc dete iminis tic F ull fec t_te c h aIs - Module 1 (Code) 


| (General) 


73 


Option Explicit 


Sub run_techs ( ) 

'Note i is the number of technology combinations and it stars in row 4' 

'Also note that j corresponds to the number of "k" factors for the RSEs for the first loop' 

Dim i , j 

For i = 4 To 2051 
For j = 1 To 16 

'set the inputs to the rse's on the sheet k_factor_RSE equal to the full factorial values on the results sheet' 

Worksheets ( "k_f actor_RSE") . Cells (5 + j, 2) .Value = Worksheets ( "Results") . Cells ( i, j + 1) .Value 
Next j 

For j = 1 To 8 

'set the inputs to the rse's on the sheet k_factor_RSE equal to the full factorial values on the results sheet' 
'note j corresponds to the # of responses and the response values will be copied starting in column T ' 

'also, the response values being picked up are from the k_factor_RSE sheet in row 3, column e (or 5)' 

Worksheets ( "Results") . Cells ( i, j + 19) .Value = Worksheets ("k_factor_RSE") .Cells (3, 4 + j). Value 
Next j 
Next i 
End Sub 
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Let’s actually evaluate all of our technology combinations. Go back to Tools and then select Macro and then 
select Macros and now select Run. You will see that the cells underneath the responses to the right start to fill out. 
The macro will continue to run until it has evaluated all the technology combinations that you provided. 



When the macro is finished, you now have the impact of ALL the technologies that you are considering on 
the responses that you were interested in as shown below. 
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Now let’s get a visual of how the sensitive the responses are to the technologies. To do so: 

• Copy all of the values of the metrics as you have done so many times 

• Open your full factorial JMP file 

• Add the appropriate number of columns 

• Paste your results 

Then, go to Analyze, then Fit Model. When the Fit Model window comes up, select your technologies and 
then hit the Macros drop menu and select the Full Factorial option. The full factorial will fill out in the “Model 
Effects” window. Then select your responses and click the Y button. Don’t forget turn off the “Center Polynomials” 
under the Model Specification drop menu. Once completed, hit Run Model. 



NUM /, 


JMP will sit there for a bit while it is calculating all the statistics. Note, the more technologies you have, the 
longer it is going to take. Once JMP is finished, minimize all of the response analysis windows and open a Profiler. 
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Again, if you have a lot of technologies, it is going to take JMP just a second to create the Profiler. Once the 
profiler is up, you now have a rapid environment whereby you can visualize the impact of ANY combination of 
technologies. As shown below. If you reset all of the technologies to a value of “-1”, this corresponds to all of the 
technologies being OFF. If you put any of the technologies to a value of “1”, then you will automatically update the 
values of the responses and see the impact that the chosen technology has on your system. Recall that all of the impacts 
to the system are inherent behind this profile. For each “-1” and “1” value in the full factorial DoE, you summed up all 
of the “k” factors and then calculated the RSEs. So, you are seeing the top level impact of all the “k” factors. You will 
never again have to run another code to determine the impact to your system from the set of technologies that you 
modeled. You can also determine if some combination of technologies will allow you to meet constraints by looking at 
the upper and lower bounds on your metrics. 
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For example, Sideline Noise (or SLN) is a response with a constraint value of 103 EPNLdB. As you can see, 
once T5 is turned on, that constraint value can be met. Additionally, if you do not set a technology to a value of 64 1”, 
this is analogous to not getting the full impact from the technology that was described in the TIM. That is, if someone 
said that I am only getting 50% of what I was expecting from the technology, then you would set that technology value 
equal to “0” on the profile and read off the value of the responses. This is a poor man’s way of handling technological 
uncertainty or changing technological assumptions. 
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One way to get around the slowness of JMP creating the Prediction Profiler or the contour plot is to determine 
which interaction terms of the regressed coefficients do not need to be calculated. This is very important when you 
have a lot of technologies, or even with the RSEs if you have a lot of input variables. So, let’s go through the steps of 
how you do that. Go back to your Fit Model window and only select one response, say TOGW. Now instead of 
selecting the “Standard Least Squares” option under the “Personality” drop menu, select the “Stepwise” option and 
then select Run Model. 



JMP will take a while doing its thing. If you would like to know that it is still running, go to the Windows 
Task Manager as shown below. JMP will consume a great deal of CPU when it is running as you can see by the “CPU 
Usage” running at 100%. When JMP is done, the CPU Usage value will significantly drop. 
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When JMP is done, the window below will come up. JMP has the ability to determine which coefficients of 
the regressed response are the most significant contributors. 



To determine the coefficients, under the “Direction” drop menu, select “Mixed” and take the default values 
that JMP gives you and then hit Go. You will see check marks appearing in the “Entered” cells. Again look at your 
Task Manager and you will know when JMP is finished by the CPU Usage significantly dropping. When JMP is done, 
click the Make Model button. 


^►JMP-FHSCT Full Factorial techs: Fit Stepwise! 
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JMP will open a new Fit Model window with the chosen coefficient terms as shown below. Don’t forget to 
turn off the “Center Polynomials” option under the Model Specification bar. If the “Emphasis” option is not on 
“Minimal Report” then select that option and click Run Model. 



The window below will come up 
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Now, go under the Response TOGW option and select “Save Columns” and then select “Predicted Formula” 



Look back 
column has the RSE 


in your DoE table and you will see a new column added called “Pred Formula TOGW”. 
behind it. 


This 
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Repeat this process for every response that you have. Now, go to Graph and select the Profiler. Then select 
all of the columns that have the predicted formulas behind them and select OK. 



Now, when the profiler comes up you can play technology games by turning on and off any technology you 
like. Also, under the Profiler drop menu, select the “Script” option and then select “Save Script to Datatable”. 
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Go back to the JMP table and you will see “Profiler” has appeared in the top left corner. What this script does 
is automatically create the Profiler based on what you just did. To execute it, right mouse click on the “Profiler” and 
select Run Script and the profiler will automatically generate. 



Populating the Decision Matrix 

When you are finished with all the technology combinations, go back to Excekl and to the “Compatible list” 
sheet. This sheet is again formatted for 1 1 technologies and 8 metrics. This sheet will automatically update from the 
values that were calculated on the “Results” sheet. On the right of this sheet is the determination of whether or not the 
technology mix (case #) is physically compatible or not. If the mix IS compatible, an “ok” is shown in the column. If 
not, then “XXXXXX” appears. A physically compatible combination is determined from the compatibility matrix and 
is coded as shown below for this example. You need to make sure that your mix of technologies is coded properly 
here. In addition, the column beside the one that determines compatibility is simply a counter. When the technology 
mix is compatible, a “1” results for the row, if not, then a “0”. At the bottom of the page is a summation to determine 
the number of physically compatible technology mixes. For this example, there are 272. 

Example compatibility rule: 

=IF(OR(AND($C4= 1 ,$F4= 1 ), AND($C4= 1 ,$ J4= 1 ), AND($C4= 1 ,$K4= 1 ), AND($C4= 1 ,$L4= 1 ), AND($C4= 1 , 
$M4= 1 ), AND($F4= 1 ,$ J4= 1 ), AND($F4= 1 ,$K4= 1 ), AND($F4= 1 ,$L4= 1 ), AND($G4= 1 ,$M4= 1 ), AND($H4= 1 ,$K4= 1 ), 
AND($I4= 1 ,$ J4= 1 ), AND($ J4= 1 ,$K4= 1 )),"XXXXXXX","ok") 
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The concepts identified in Step 6 (i.e., only the compatible technology mixes) form the rows and the system 
metrics from the problem definition form the columns as shown below. The deterministic elements of the matrix are 




Note since you evaluated your technologies deterministically, you actually have your Decision Matrix 
defined. It is the matrix defined by the “compatible” list of technologies and the corresponding metric values in the 
“Compatible List” sheet in the “Calc deterministic Full fact tech” spreadsheet. Let’s create our Compatible list of 
technologies based on the full factorial. Place your cursor in cell B4 and select everything including the compatibility 
rules in Column W, as shown below. 


E* Beta Y»ir U»«n Fcnmi Took D*» Statoir C*1 U«lp 

o & u aar x fc <f - * - - «. * * a u it « * ®; 
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Now go to sheet “Compatible List” and Paste Special” and paste only values starting in cell B4. You should 
have a complete mirror of what the “Check Compatibility” sheet had except there are no formulas. Now, you want to 
sort this information based on whether or not the technologies are compatible. So, keep the area that you just pasted 
highlighted and go to Data and select Sort. When the window below comes up, you want to select the option for “My 
list has” no header row and then sort by Column W in Descending order. This will sort with all of the compatible 
technologies listed first and then the incompatible ones below that. 



And you will see that the order of the technology combinations changes, as shown below. Now you can 
simply remove all of the incompatible cases. This should leave 272 compatible combinations that we will take to the 

selection step. 

KIWifa 



Ready 


Sum =2098448 
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Probabilistic Technology Evaluation 

All of the evaluation of the technologies so far has been deterministic. This assumes that the technologies will 
reach the maximum possible impact, the values found in the TIM. Since these technologies are not fully matured, there 
is a chance that the final outcome will be less then the value in the TIM. To account for this variation in final outcome, 
we can use probabilistic evaluation. The file “Probtech eval” will allow you to evaluate the impact of technologies 
probabilistically. This file contains the sheets “Definitions”, “Prob Analysis”, “Cases”, and “RSE”. There may also be 
response sheets (called Rl, R2, etc.). If there are not, they will be created later. Go back to the “Compatible list” sheet 
in the “CalcdeterministicFullfacttech” file. Copy the case numbers and technology indicators for the compatible 
cases. 


||S Ble Edit View Insert Format Tools Data Window Cell Run CBTools Help - Ifi 1 ! x|| 

□ g» b nz i (ta # [# | 65% - 1 o 
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These are the cases you want to run probabilistically. Paste them into the “Cases” sheet by highlighting cell 
A10 and pasting the cases. 
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Bring your technology impact matrix into the “Prob Analysis” sheet. The ‘Probabilistic Scale’ matrix below 
the TIM will update automatically from the TIM and the TRLs. The scale is used in defining the distribution and 
follows the formula: 


Scale = |(0.3 * impact ) - (TRL - 1)| * 


||0.3 *impact\ - 10.05 *impact\\ 
8 


If you have more technologies or “k” factors, you will need to expand both the TIM and the scale matrices. 
You must change all of the values in the technology impact matrix to negatives in order to run the simulation because 
you will use Weibull distributions to define each impact and this distribution must have a negative reference. The 
Weibul distribution is used because it best models the possible impacts of the technology by incorporating the TRL 
into the scale of the distribution. 
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To define these distributions, highlight all of the cells in the TIM that contain influences (shown in green 
above). Click on the Cell menu, then Define Assumptions. This menu will pop up: 



Select Weibull and then OK. 
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TIES for Dummies 


In the next menu, you should rename your assumption. Then check that the location and left bound are the 
impact from your TIM. The shape should be two and the right bound should be ‘+Infmity\ The scale will come from 
the probabilistic scale matrix. The cell can be referenced by entering ‘=B27’ in the menu. The scale should remain 
static through the simulation. Repeat this as each pair of windows pops up. Be sure to reference the scale for that 
specific technology and “k” factor. 
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Now, go back to the “Cases” sheet. The cells that contain the normalized “k” factors must be updated. These 
are linked to the “Prob Analysis” sheets so they will change with the changing tech combinations and “k” factors 
distributions. This is where the negatives entered into the TIM must be corrected. If a value in the TIM was changed 
to be negative, add a negative before its reference in the equation in the appropriate “k” factor as shown below. 
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Any necessary negatives go here. 
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TIES for Dummies 


Add in your RSE on the “RSE” sheet. The ‘actual values’ for the “k” factors must be linked to the “k” factor 
cells you just checked on the “Cases” sheet. The responses should be highlighted and defined as the forecasts under the 
Cell menu and Define Forecast. 



Rename the forecasts. 


Cell E3: Define Forecast 


Forecast Name: [ uiIekYJ 
Units: P 


xj 

If this is not the response name, 

4 - 

change it. 
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TIES for Dummies 


The “Definitions” sheet defines the necessary information in order to run the macro to assess the different 
technology combinations probabilistically. The ‘Inputs’ reflect the number of technologies, how many Monte Carlo 
repetitions you want, how many responses, and how many cases. The ‘Run Preferences’ set the run preferences for the 
macro. The setup in this shot is the fastest way to run. Your computer’s Run Preferences will remain as set by the 
macro after it has completed. To change them, go to Run, Run Preferences. The Run Preferences will change to the 
preferences on the “Definitions” sheet every time you run the macro. 
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TIES for Dummies 


Unless you have changed sheet names, the macro should not require editing. Under Tools, choose Macro, 
then Macros. Select Monte_Carlo and Run. This macro will run Crystal Ball on each technology combinations. It 
extracts the percentiles and statistics and pastes them into the sheet for each response (R1 is the data for the first 
response and so on). If you want to see what it is doing, edit the macro and change all the Application. ScreenUpdating 
= False to true. This takes longer, but you can see each part of the macro. 



A sheet for each response, numbered in the order of the RSE, has been created and filled by the macro. Your 
response sheets contain the percentiles and the statistics for each case. 
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These response sheets form a second decision matrix with probabilistic instead of deterministic results for 
each metric. Only compatible cases were brought in and evaluated so the cases do not need to be tested for 
compatibility as was done with the deterministic decision matrix. The probabilistic decision matrix will be used in step 
8 to examine Technology Sensitivities and Technology Frontiers for different confidence levels. 
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Step 8: 5ejecf_ Besf_ Family_ of Alternatives 

For any multi attribute, constraint, or criteria problem, the selection of the “best” family of alternatives is 
inherently subjective with no single answer fulfilling all requirements. Four techniques are used in the TIES method: 

1) Multi- Attribute Decision-Making techniques (TOPSIS) 

2) Technology Frontiers 

3) Technology Sensitivities 

4) Hierarchic Genetic Algorithms 


Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) 

A Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) is utilized to down select the 
proper mix of technologies satisfying the system level metrics. TOPSIS provides a preference order of the 
deterministic values obtained in the Decision Matrix, at a given confidence level, resulting in a ranking of the best 
alternative concepts. I will not go through the math behind the TOPSIS method. You can go read one of the references 
to get more info. I will simply explain how you do it with the spreadsheets that I have provided. 

If you have the HSCT with 1 1 technologies and the compatibility matrix from Step 6 of this tutorial, you have 
only 272 compatible combinations (or alternatives). You have obtained your response values of the vehicle with those 
technologies “on”. Open the Excel file “TOPSIS tech ranking”. The window below will come up, just click the button 
Enable Macros. 



Regardless of which TOPSIS spreadsheet you have, there are 8 sheets including “Inputs”, “Scenarios”, 
“Alternatives List”, “WeightednormalizedDM”, “Euclidean dist”, “Rankings”, “Calc Area for Chart”, and 
“Radargram for subset”. 
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The first sheet is the “Inputs” sheet, but we will come back to it. On the next sheet, “Scenarios”, are 10 
different subjective weighting scenarios. What I typically will do is place heavy weighting on the performance metrics 
on the left and shift to heavy economic weighting on the right. I tend to place more importance on the metrics that are 
the concept “show-stoppers”, or constraints that are hurting me most. In the sheet below, one scenario that I would 
consider is one that places heavy weighting on the Flyover and Sideline Noise metrics, since both of those had very 
low, if even existent, feasibility values. Hence, insert 10 or more different weighting scenarios depending on the 
metrics, the significance of each metric, and try to capture varying importance of the different metrics. The idea here is 
to determine if a given set of technology mixes will dominate in importance regardless of the weighting scenario 
considered. 
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The “Alternatives List” sheet has the case numbers of the compatible technology combinations and the 
corresponding metric values for 8 metrics. You need to copy the metric values and case numbers into this sheet for all 
the compatible cases. You can copy these from the “Compatible list” sheet in the “Calc_deterministic_Full_fact_tech” 
file. As shown below, the square root of the sum of squares is calculated for each metric as the bottom of the page. The 
equation for this is shown for the TOGW below. 

Example Excel formula for square root of sum of squares (TOGW): 

=SQRT(SUMSQ(C$4:C$275)) 

This format is used for all metrics. Note, if you have more or less responses than the 8 given in the HSCT file, 
you need to modify ALL sheets that are dependent upon the number of response (i.e., attribute) columns . 
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The next sheet is the “WeightednormalizedDM”. Two things are done on this sheet. First, the decision 
matrix is normalized and multiplied by the subjective weighting values defined by you. It is updated automatically 
from the alternatives sheet. It is found with this equation: 

=( 'alternatives list'! original value/'altematives list'! sum-of-squares )* weighting value 

Next the positive ideal and negative ideal solutions are determined for each metric. These are at the bottom of 
the matrix and are determined based on the following equations. For both TOPSIS spreadsheets provided, all the 
metrics are desired to be minimized, i.e., in the context of TOPSIS, they are considered “costs”. If you want to 
maximize a metric, it is considered a “benefit”. So, the Excel formulas used to determine the positive and negative 
ideal solutions are: 

If you want to minimize your metrics, say TOGW, the ideal solutions are defined as: 

Positive Ideal Solution, S + =MIN(C5:C276) 

Negative Ideal Solution, S" =MAX(C5:C276) 

If you want to maximize your metrics, say TOGW, the ideal solutions would be defined as: 

Positive Ideal Solution, S + =MAX(C5:C276) 

Negative Ideal Solution, S" =MIN(C5:C276) 
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The next sheet is the “Euclidean_dist” sheet. This sheet determines the separation from the positive and 
negative ideal solutions and calculated by the difference of the sum of squares of the weighted decision matrix and the 
positive ideal solution and negative ideal solutions, respectively. Examples of these two calculations are provided. In 
addition, the relative closeness to the Ideal Solution is determined. You will use these values for the alternative 
rankings. An example of how this is calculated is also provided. 

Example formula for the Separation from the Positive Ideal Solution: 

=SQRT(SUMXMY2(Weighted_normahzed_DM!C5:J5,Weighted_normalized_DM!C$277:J$277)) 

Example formula for the Separation from the Negative Ideal Solution: 

=SQRT(SUMXMY2(Weighted_normahzed_DM!C5:J5,Weighted_normalized_DM!C$278:J$278)) 

Example formula for calculating the Relative Closeness to the Ideal solution: 

=E4/(E4+C4) 

Remember, if you have a different number of cases or metrics, you need to modify the cell referencing. 
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The next sheet is “Rankings”. This sheet has columns for the ranking, case number, and the ‘relative closeness 
to ideal solution’ value, which is labeled ‘Ranked Order from Best to Worst’. The case number and ranked order 
columns should be blank until you run the macro. 
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The last two sheets we’ll talk about later. So, let’s run the macro. 
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To evaluate the different weighing scenarios, you can run the “run_topsis” macro. In order to do this, you 
must fill in the “Inputs” sheet to reflect the number of technology combinations, responses, weighting scenarios, and 
top alternatives to extract. Then go to the Tools menu, select Macro, and then Macros. Select the “runtopsis” macro 
and hit Run. 

The first thing the macro does is copy the weighting factors for the first scenario from the “Scenarios” sheet to 
the “Weighted_normalized_DM” sheet. The values for the weighted normalized responses automatically update. Then 
the values for the ‘Relative Closeness to Ideal Solution’, which also update automatically, are copied from the 
“Euclideandistance” sheet and pasted to the “Rankings” sheet under the ‘Ranked Order from Best to Worst’ column 
and to the “Calc Area for Chart” sheet into the column for the first weighting scenario. The case numbers are then 
copied to the “Rankings” sheet in the ‘Case’ column. Next, the ‘Case’ and ‘Ranked Order from Best to Worst’ 
columns on the “Rankings” sheet are sorted in descending order by the ‘Ranked Order from Best to Worst’ column. 
Finally, the top 25 case numbers are pasted into the “Scenarios” sheet in the ‘Ranking’ column for the first scenario. 
This process is repeated for all the weighting scenarios. 

The “Calc Area for Chart” sheet uses the relative closeness values to find a top ten overall. This page 
calculates area based on a ten spoke circle (based on ten weighing scenarios). It calculates the area, sums it, and uses 
this to rank its top ten combinations. These top ten are graphed, with their relative closeness values for each weighting 
scenario, on the last page, “Radargram for subset”. This graph should automatically update when the macro has been 
run. The “Calculated Area for Chart” sheet should look like this: 
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0.385527723 0.060234443 
0.380664028 0.058728518 

0.011611024 

0.011177917 
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0.164735861 

0.236404352 

0.165661669 
1 0.235521669 

0.368909189 

0.354388086 

0.333916798 

0.303318998 

0.382707004 

0.366575418 

0.409401194 

0.364744287 

0.184544275 

0.176820365 
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0.165661669 0.050906181 
0.235521669 0.073052962 

0.004216601 

0.008602778 

0.368909189 0.051192271 
0.354388086 0.072780198 

0.00944265 

0.012896218 
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5 
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0.0843789 

0.412109354 

0.086831682 

0.434755712 

0.377976267 

0.293306079 

0.365249609 

0.321289462 

0.387713522 

0.237858568 

0.417010389 

0.361409361 

0.175150254 

0.171279092 

5 

325 



0.086831682 0.026074514 
0.434755712 0.127348794 

0.001132047 

0.027682808 

0.377976267 0.026832465 
0.293306079 0.134346903 

0.005071018 

0.019702382 
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1 

0.073228476 

0.075664285 

0.365242865 

0.34330774 

0.382980695 

0.405403974 

0.169720027 

1 



0.075664285 0.022628843 

0.000856098 

0.365242865 0.02338155 

0.004269972 
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0.405653018 

0.224195994 

0.214305681 

0.428638589 

0.224611164 

0.214987607 

0.280930325 

0.344099883 

0330578931 

0.301730962 

0.326915342 

0.302955654 

0.227185238 

0.344236339 

0.33961365 

0.348817033 

0.367264992 

0.355292245 

0.165443674 

0.158978309 

0.154145496 
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0.428638589 0.125353676 
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0.026865711 
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0.00711868 
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0.344099883 0.069408667 
0.330578931 0.066434824 

0.018605539 

0.011941757 

0.010980977 
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0.120175878 

0.121880774 

0.326815431 

0.309302059 

0.328176901 

0.372751722 

0.137196322 

261 



0.121880774 0.037136389 

0.002263106 

0.326815431 0.037663231 

0.006154462 

272 
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0.110481501 

0.112211564 

[0313147129 

0.284014607 

0.321895049 

0.360211557 

0.131938232 

257 



0.112211564 0.034140661 

0.001915489 

0.313147129 0.03467528 

0.005429232 

276 












H 

277 

M 4 

► N \ Inputs / Scenarios / Alternatives List / 

Weighted normalized DM 

zuium 









: & AutoShapes * \ s * □ O H -41 <$* ’ -JL " C 


M. Kirby 


Page 96 


TIES for Dummies 


And the Radargram like this: 
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Technology Frontiers 

The inefficiencies of the Multi-Attribute Decision-Making techniques, deterministics, and non-intuitive 
numerical results may be improved with the use of the Technology Frontiers. Technology Frontiers are defined as the 
limiting threshold of an “effectiveness” parameter. The technology frontiers are similar to TOPSIS with the use a user- 
defined function for which maximization is desired. The Technology Frontiers approach involves calculating the 
parameters for Performance Effectiveness (PE) and Economic Effectiveness (EE) using the baseline values and the 
value of the metrics for each alternative. 

Open the file “Tech Frontiers”. This sheet will graph the technology frontiers for performance and economic 
effectiveness at each confidence level. This file has many sheets, including “total data”, “sorteddatatheo”, 
“sorteddatalO”, “sorted_data_50”, “sorted_data_90”, and “Calc_fronts”, plus sheets for each of the graphs. First, 
you need to copy the compatible case numbers and Is and -Is on the “total data” sheet. Next, copy the value for each 
response from the “Compatible List” sheet of the “Calc deterministic Full fact tech” file and paste it into the 
‘Theoretical’ column for each response. Now go back to the “Prob tech eval” file and click on the first Response 
sheet. Since the “Tech Frontiers” file needs the response values for the confidence levels of 10%, 50%, and 90%, copy 
these columns from the response sheet into the appropriate columns on the “total data” sheet. To clarify’ for the first 
response, which is TOGW on sheet “Rl”, copy columns D, L, and T and paste them into columns O, P, and Q. Repeat 
this for all the responses. Be sure to modify the number of responses if you have more or less then eight. If your 
number of cases differs, change cell references as you move through the workbook. 
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Copy these columns 
and paste them here. 
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Now scroll to the right. The columns after the response columns contain formulas to calculate feasibility and 
performance and economic effectiveness. Check the formulas in columns BE through BH (feasibility check) and 
change them if you have different constraints. When all the formulas are updated, place your cursor in cell A4 and 
highlight all cells from B4 to BH275. This should be 272 rows, or as many cases as you have, and all columns from the 
‘Original Case Number’ column to the ‘Performance Check for Technical Feasibility (90% Confidence)’ column. 
Copy this and go to the next sheet, “sorteddatatheo”. Paste Special and choose Values into cell B4. On the 
“sorteddatatheo” sheet, sort by column "BE" in descending order, then "AU" in ascending order. Copy the original 
data from the “total data” page into the “sorteddatalO”, “sorted_data_50”, and “sorted_data_90” sheets and follow 
the sorting instructions in yellow at the top of each page. 


X Microsoft Excel - Tech Frontiers 
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Now check the 16 graphs that follow. These are plots of RDT&E versus the various effectiveness parameters. 
These parameters are automatically calculated in the various sheets. If you have different variables or want a different 
weighing scenario, these formulas must be updated. The basic forms for these equations are: 

PE Ak = g rc G »v TOFU Vm^ +s m^ +s SH^ 

' TOGW Mi f TOFL Ah * Vapp F£W,„ SLN Mi 


EE Al =a + fi S,SPM *i 

' r $IRPM A „ } 

The graphs should automatically update based on the new data you copied in and sorted, but you need to 
change the cell range for each number of technologies. Click on the first graph page, “perf-theo”, right-click and 
choose ‘Source Data’. On the ‘Series’ tab, click on each of the data series and adjust the x and y values based on the 
number of technologies. 
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Now go to the “Calc fronts” sheet, which calculated the thresholds. To calculate the Performance threshold, 
use the formula: 


PE threshold ~ a 


TO GW, 


BL 


_ tofl bl 
■ + /? BL 


+ Z 


Vcipp BL 


+ 5 - 


FON 


BL 


- + 8 - 


SLN 


BL 


1,000,000/fe 11,000/? 155 kts 10 6EPNLdB 103 EPNLdB 


This formula should include all of your performance metrics. If you are using different constraints or metrics, you will 
need to adjust the formula. The weighing values are arbitrary. For this example, the different metrics have been 
weighted evenly. The coefficients must sum to one. 

For the economic threshold, the formula is: 


EE 


threshold 


Acq% m $/ RPM rj 

a — -^+6 — 

$185M $0.10 


This reflects your economic metrics. If your metrics or constraints vary, the formula must be updated. For 
this threshold, $/RPM has been weighted with 0.75 in order to place more importance on that metric. The PE 
Threshold is the average of the performance and economic thresholds. 

The Technology Frontier must be added using drawing tools. You can adjust an existing frontier by right 
clicking on the shape and selecting Edit Points. This will allow you to move and reshape the frontier. To add a new 
frontier, select AutoShapes from the drawing toolbar. Select Lines and then the curve as shown below. 


11*0 File Edit View Insert Format Tools Chart Window Help 
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You can now drag and click points around the data points to make the technology frontier. Repeat checking 
that the number of technologies and the technology frontiers are correct on the rest of the graphs. 
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For the graphs of Economic versus Performance Effectiveness, check the data for the ideal point. This point 
should come from the maximum values of the two data groups at the given confidence level. 
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One interesting comparison is the Technology Frontiers between the confidence levels. This shows how the 
different confidence levels affect reaching the threshold without changing the general shape of the frontier. 
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Technology Sensitivities 

A technology sensitivity investigation is performed by a comparison of the infusion of the individual 
technologies to the conventional configuration, and evaluation of the deviations in metric values. The idea here is to 
determine which of the different technologies, as applied in isolation of the others, most influences the vehicle metrics. 
You have all the data and can simply manipulate it and format some pretty pictures. The example below is the impact 
of the 11 individual technologies on the TOGW of an HSCT. As you can see, T4, which was the HLFC technology, 
had the most significant impact on reducing the TOGW from the baseline value. Whereas T5, which was the 
Environmental Engines technology, had the most significant NEGATIVE impact on the TOGW. 


Deviation from Baseline Takeoff Gross Weight (%) 
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Now open the file “Tech Sensitivities”. This file will calculate and graph the technology sensitivities. The first 
page is the “Data” page, where you need to copy in the technology impact matrix into the graph starting in cell B4. 
Then update the formulas for the “k” factors in the second graph starting in cell B18. Change the baseline in cells B45 
to 145 if it differs from the one given. Be sure to edit the number of columns and rows if you have more or less 
technologies, metrics, or “k” factors. Next, you need to update the “RSE” page for your RSE. Then go back to the 
“Data” page and select the Tools menu, then Macro and then Macros. Choose the “runtechs” macro and edit it to 
reflect the number of technologies and responses. Make sure the cell references are right. Now run the macro. The cells 
from B32 to 142 should fill in with the metric values, and the ‘Percent Change from Original Baseline’ table should 
automatically update. Once the macro has finished, you can edit the graphs for the metrics. 

This information can also be found from the “Calc_deterministic_Full_fact_tech” spreadsheet. Find the case 
that has only the tech you are interested in on. Copy the metrics from these cases into the ‘Metrics’ matrix. This will 
update the ‘Percent Change from Baseline’ and the graphs. 
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109.9005 
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191.890066 

14577.22 

0.10747091 

















.34 

T3 

858716 

10718.77 
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Here’s a sample technology sensitivity graph of approach speed. You can see that T5, environmental engines, 
increased the baseline the most, while T3, which is circulation control, reduced it the most. 

Approach Speed 



Using the “Probtech eval” sheet, it is possible to graph the different confidence levels for each technology 
sensitivity. Open the file and save it as “Prob each tech”. Now change the tech combinations on the “Cases” sheet so 
that each row has one tech on and the rest off. This chart should have as many rows as you have technologies. 



Now, change the number of cases on the “Definitions” sheet and rerun the Monte_Carlo macro. 
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Open the “Tech Sensitivities-prob” file. Copy the data from each response page into the probabilistic 
evaluation section. 



The percent change and the graphs will automatically update for the new data. Here is what the graph for 
Approach Speed looks like now with the confidence levels: 


Approach Speed 



Percent Change from Baseline 
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Genetic Algorithms 

Genetic Algorithms (GA) offer a unique way of selecting the best technology combination by simulating the 
natural processes of breeding and competition. Random combinations of technologies are created with an array of l’s 
and 0’s representing the techs in the on and off configurations respectively. Each combination representing all the 
technologies in either the on or off position is called a chromosome. A certain number of chromosomes are randomly 
created within a population and then the RSE's are used to obtain the response for each combination. The responses are 
normalized by some kind of “fitness function” which allows one number to represent the goodness of the combination. 
A combination with a low fitness value is closer to the ideal since the minimization of the metrics is desirable. The 
population is then put through a reproduction scheme whereby the more “fit” a combination, the better the chance that 
it will be selected for reproduction and its offspring will become more dominant in the next generation. In addition to 
competition and reproduction, the concept of mutation is introduced into a population in order to capture effects that 
might not have been available in the first random selection. Thus after several generations the most fit combinations 
will have achieved prominence and the “best” combination can be found. 

When a large number of technologies exist the only way to use JMP and the TOPSIS method is to break the 
techs down into groups of 13 or less and analyze them separately. This, however, does not capture all the interactions 
between technologies. The best use of genetic algorithms comes when the number of available technologies is greater 
than 13, which is the largest full factorial combination that JMP can create. GA is run through a Matlab code that is 
very modular and must be set up for each concept that is investigated. This is done by creating certain text files that 
contain the information needed by the program. 

An example GA run for a 225 passenger subsonic passenger jet is shown here. The background information 
for this jet is found in the “225 Pax Info” Excel file. This concept has 36 possible technologies for infusion, which 
would require three separate groups under TOPSIS. Here we will analyze them all together using GA's. The first and 
most complicated task is to determine patterns within the technology compatibilities. Because each combination is 
compared to the others within each population and the next generation based on this result, it is imperative that only 
compatible combinations be used in competition. 


Table 8: Technology Compatibility Matrix 
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By further analyzing this compatibility matrix, it would become evident that all the incompatible 
combinations fall within groups of technologies that are compatible with any technology outside the group but 
incompatible with anything in the group. These groups are shown in Table 9. The 12 remaining technologies are 
compatible with all other technologies under consideration. 

Table 9: Technology Groups 
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The original organization of the TIM had nothing to do with which techs were in which compatibility group 
but the Matlab requires that the chromosomes created for it be organized by these groups. Therefore, it is necessary to 
rearrange the values in the TIM. The order needed is given below. 
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Open the Technology Impact Matrix and rearrange the technologies in order. Be sure to put in zeros where the 
technology does not have an effect. Copy just the impacts and paste them as values into a new Excel sheet. 
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Save this file as the TAB delimited text, named “TIM.txt” as shown below. 
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Now create the other input files. Copy the RSE’s from JMP into EXCEL, then remove the titles in the first 
column by removing column A. Save the file as TAB delimited text under the file name “RSE.txt”. The RSE’s need 
the inputs to be in the same form as FLOPS so the “k” factors must be converted in order to be useful. The converting 
factors and multipliers can be found in Table 10. Most of the values needed are factors distributed around 1, where an 
increase in the value leads to a number greater than 1 and a decrease leads to a value less than 1. Some of the economic 
factors, however, are based around 0. Other variables are needed as dimensional values and not factors (it will depend 
on which variables you are using, check the FLOPS manual for more information). The Matlab program will convert 
the “k” factors given the factors and multipliers in Table 10. You should copy and save the numbers in this table as 
TAB delimited with the filename “ranges.txt”. 


Table 10: K Factors and Baseline 



Minimum k 

Maximum k 

Factor 

Baseline 

Multiplier 

Wing Weight 

0.65 

1.15 

1 

1 

Fuselage Weight 

0.75 

1 

1 

1 

HT Weight 

0.6 

1 

1 

1 

VT Weight 

0.6 

1 

1 

1 

Cdi 

0.8 

1 

1 

1 

Cdo 

0.8 

1 

1 

1 

LG Weight 

0.75 

1 

1 

1 

Avionics Weight 

0.5 

1.05 

1 

1 

Hydraulics Weight 

0.5 

1.05 

1 

1 

Fum. and Equip. Weight 

0.9 

1.05 

1 

1 

VT Area 

1 

1.8 

1 

250 

HT Area 

1 

1.888888889 

1 

450 

Engine Weight 

0.55 

1.05 

1 

12126 

Fuel Consumption 

0.8 

1.01 

1 

1 

RDT&E Costs 

-0.2 

0.2 

0 

1 

O&S Costs 

-0.2 

0.2 

0 

1 

Production Costs 

-0.2 

0.2 

0 

1 

Utilization 

0.8 

1.2 

1 

4915 

Wing Area 

0.82111437 

1.026392962 

1 

3410 

Thrust-to-weight ratio 

1 

1.166666667 

1 

0.3 
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In order for the chromosomes to be compared, there must be a number that represents the fitness of each. This 
number can be chosen from a variety of possibilities based on what metrics are of greatest interest in that case. For this 
example a comparison between several metrics and their constraint values will be used as the measurement. The 
equation for the fitness factor is: 

F responsej response 2 + 
constraint! constraint 2 

where alpha is the first weighting value, beta is the second weighting value, and so on. The constraints are stored in the 
order of this equation in a text file called “base.txt” with the weighting values for the metric located in the row beneath 
them. In this example, there are five metrics with constraints that are significant, which are C02, $/RPM, TAROC, 
DOC+I, and NOx. These constraints and weighting values can be entered into two lines of EXCEL and saved as TAB 
delimited text. The Genetic Algorithm programs are set up so that minimization is desirable. If you wish to maximize 
your responses, it is necessary to modify the “run225.m” file. To do this, add the following line of code after the last 
existing line: 

F_value = 1 / F_value 

This is required because the “tournament. m” file is set up so that a smaller fitness value is preferable. 

The Genetic Algorithm code is not a single program; instead it consists of 21 separate program files. These 
should all be in one folder called “Matlab Files”. This folder should also contain the various input files you created 
above. It is essential to set up Matlab to recognize wherever you have the program and input files. To do this open the 
File menu, go to Set Path then Add Folder and add the “Matlab Files” folder where the files are located. Once all the 
input files are correctly set up all that’s left is to run the code. Make sure that there are no files in your work directory 
that duplicate the names used for this program. If there are, these files will be used instead of the correct ones. If you 
plan on running a different number of runs, be sure to change the “hgat36.m” file by changing the ‘RUNS’ line in the 
‘Input’ section. To run the Genetic Algorithms codes, type ‘hgat36’ in the Matlab command window. The entire 
Genetic Algorithm process will run from this code. 
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Once the program is finished running for the number of runs selected, it is just a simple matter to analyze the 
output and determine the best chromosome. Open the ‘grandbesf variable in the Matlab Workspace window. 



Ready 


The number that this gives you represents the best fitness that any technology combination was able to obtain. 
Since there were four populations run simultaneously, it is necessary to determine from which of these populations this 
fitness value comes from. This can be done by opening the ‘best_fif variables and determining which matches the best 
overall value. Then open the ‘bestchromosome’ that corresponds to the best fit; this is your best technology 
combination. Note that the order of the “on” and “off’ tech switches corresponds to that of the Technology Impact 
Matrix. 



M. Kirby 


Page 112 








TIES for Dummies 


Modifying the Code 

For a different vehicle and technologies this code must be significantly changed in order to be used. The most 
dificult aspect will be setting the code to only pick compatible cases. If your vehicle has technologies that can be 
organized into groups such as in the previous example where A f B, A f C, and B f C then the changes to the code are 
minimal. However, for technology sets where A f B, A f C, but B = C then a whole new approach must be sought. 

For the former case where the groups are different but still all the techs can be categorized into one then the 
procedure is rather straightforward. First let’s look at the initial random population selection. Open the file “hgat36.m” 
and scroll down to lines 69-82. 



In this area of the code change the 12 to the number of completely compatible technologies in your study. It 
will also have to be changed in lines 87, 181, and 234. In order to modify the code for the selection of techs in each 
group open the file “tail.m” 
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The first thing that is evident is that there are only 2 calculations going on. This is due to our previous 
example having only 2 different sized groups- groups 1&2 with 6 techs each and groups 3-6 with 3 techs each. If your 
case has more variance in the size of the groups you will need to create new nested ‘for’ loops. The overall method of 
this code is blatantly simple: an identity matrix is formed with each row representing a case with only 1 of the 
technologies turned on. There is an equal probability for any of these rows to be selected along with the same 
probability that none will be selected which represents the case of no technologies from that group being activated. In 
order to modify the code the ‘for’ loops must be changed to represent the number of techs in each group. At the end of 
the file the selected configurations are combined to form the last section of the chromosome. This must be changed as 
well to correctly piece together the selections in your new technology set. 

This takes care of the initial population creation but the later mutation of the chromosome must also be 
altered. Mutation is taken into account in order to possibly allow in technologies not selected in the initial population. 
The mutation code can, however, allow in technologies that are incompatible with those already selected. For this 
possibility another code is run in order to screen for this effect. Open the file “genemanipulation.m” and look at lines 
5-16. 


%look at 13, 14, 15, 16, 17 

if sum(pop(i,13:18)) > 1, 
loc = fmd(pop(i,13:18)),* < 
if rand < 0.49, 

pop(i,loc(l)+12) = 1 
elseif rand <0.98 
pop(i,loc(2)+12) = 1 
else 

pop(i,loc(l)+12) = 0; 
pop(i,loc(2)+12) = 0; 
end 


18 


13-18 represent section of 
the chromosome being 
affected 


pop(i,loc(l)+12); 

pop(i,loc(2)+12); 


This section of the code looks through the part of the chromosome containing group 1 and determines if more 
than 1 of the technologies in the group is on and, if so, there is a 49% chance of the first tech being selected, a 49% of 
the second tech being selected and a 2% chance of both being turned off. This is run through several times for each 
group to ensure that if more than only 1 tech and most remains on. In your code you must edit these sections of the 
code in order to look at each section of the chromosome representing incompatible technologies and make a proper 
selection. Now that the chromosomes are correctly set up the rest of the code will run without needing any changes. 

There are a few things that can be changed should you choose to do so such as the equation for fitness used by 
the algorithm to compare the chromosome in competition. This can be done in the file “run225.m” on line 44. This 
equation is simple to edit but you must remember that it and the file “base.txt” are linked so changes to one necessitate 
changes to the other. Changing the RSE’s and TIM’s can be done just as described in the previous 225 passenger 
example. 
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If you have any questions or comments, you can reach Dr. Kirby at 
michelle.kirby@aerospace.gatech.edu. 
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Appendix A: Parse. ShelJ_ Script Description 

################################################################################################# # 
# # 

# PARSE # 

# # 

# Written by Samir El Aichaoui # 

# Supervised by Dr. Mark A. Hale # 

# Aerospace System Design Lab # 

# Georgia Institute of Technology # 

# Summer 1998 # 

# # 
################################################################################################# # 

################################################################################################# # 

# DESCRIPTION # 

# # 

# PARSE allows a user to extract data from a formatted data file. We have found it to be # 

# particularly useful for use with automating analysis, "wrapping", programs for use in # 

# software architectures. # 

# # 

# PARSE is command line driven and uses tk/tcl as its core. # 

# # 
################################################################################################# # 

################################################################################################# # 

# ARGUMENTS # 

# # 

# This program supports the following command-line options : # 

# -search allow you to define the search criteria that will define a certain position # 

# (or more specifically line) in the file, this option requires a string # 

# as an argument. This search string need to be present in the file # 

# This option is required. # 

# -read specifies which word from the line you want to extract. When this option # 

# is not specified the default 1 is used. # 

# -forward start the search for the search criteria from the beginning of the file. # 

# -back start the search for the search criteria from the end of the file, this is # 

# useful when the results sought are at the end of the output file. The # 

# default value is -forward. # 

# -occurance defines the number of times the search criteria occur in the file before # 

# reaching the line considered, the default value is equal to 1 . # 

# -offset defines the number of lines offset with respect to the line defined by the # 

# search string, the default value is equal to zero. # 

# -split stringl after string2 is used in the case of values following string2 # 

# ( example = ) and separated by stringl ( example , ) . # 

# -matrix inti int2 allows to extract a matrix instead of a single string, inti defines # 

# the number of rows and int2 specifies the number of columns. # 

# # 
################################################################################################# # 


M. Kirby 


Page 116 





TIES for Dummies 


################################################################################################# # 

# USAGE # 

# usage: parse98 -search string [-read int] [ -forward | -back] [-split char after char ] # 

# [-matrix int int ] [-occurance int] [-offset int] [-help] file # 

################################################################################################# # 

################################################################################################## 

# EXAMPLES # 

# # 

# 1) parse98 -search "something" input.dat # 

# will search for the first occurrence, first word in the line, # 

# from the beginning # 

# of the file input.dat for the word "something" # 

# the value returned will be the first value on the line # 

# # 

# 2) parse98 -search "something" -back input.dat # 

# will search for the first occurrence, first word in the line, # 

# from the end of the file input.dat for the word "something" # 

# the value returned will be the first value on the line # 

# # 

# 3) parse98 -search "something" -read 3 -matrix 3 3 -back -offset -2 -occurance 3 \ # 

# input.dat will look for the matrix found two lines before the # 

# line containing "something" in its third occurrence in the file. # 

# The matrix will have three rows and three columns and will be # 

# starting in each line at the third word. The file used is # 

# input.dat. # 

# # 

# 4) parse98 -search "something" -read 3 -split "after =" input.dat # 

# if the line found is " something=3 . 4 , 5 . 6 , 7 . 6" # 

# then the result returned is 5.6 # 

# # 
################################################################################################# # 
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Appendix B: T5 W Program Guide 


Aerospace Systems Design Laboratory 
School of Aerospace Engineering 
Atlanta, GA 30332-0150 
www.asdl.gatech.edu 


March 12, 2001 


Background 

TSW is a UNIX utility program that substitutes variable values into a namelist formatted input file. 

Files 

There are three main files of interest when using TSW for file substitution. The files can have any name but 
all three files must have unique file names. The general file layout is shown below: 

Input File Substitution Template 


I * 


TSW 

f 


I 

Output File 


N 

Input File 

The input file must be namelist formatted input file in which variable values are to be substituted. A sample 
input file follows: 




$OPTION 

IOPT=l, IFITE=1, 

IANAL=3 , 

$END 

$CONFIN 

DESRNG=8 0 0 .0, TWR=0 . 63, 

GW=30000 .0, 0.0, 25000.0, 35000.0, 
$END 


The following rules apply: 

• Namelists must be all $ or all & separated 

• Variables may be scalar or array 

• Tabs may not be used in the file (e.g. at the beginning of the line) 

• Variables must be comma separated and the last variable in a namelist must have a comma after it 
Substitution Template 

The Substitution Template defines what variables are to be substituted into the namelist and what the new 
values are. The general file format is: 


namelist variable value 
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namelist variable value 


The following rules apply: 

• All variable and namelist names must appear exactly as they are in the input file 

• If a scalar variable does not appear in the given namelist, it is appended automatically to the end. Array 
variables cannot be appended. 

• Array variable values are indicated by []. For example, the third element of the variable GW is indicated 
by GW[3]. 

• A can be used to apply the substitution to all namelists. 

A sample substitution template follows: 


OPTION IFITE 2 
OPTION IENG 1 
CONFIN DESRNG 750.0 
CONFIN GW [3] 28000.0 


Output File 

The output from the file parsing is put into the output file. 

The following rules apply: 

• The output filename cannot be the same name as the input filename 

• The file is organized for single variables to appear on their own line 

• Formatted text is ignored 

A sample output file follows for the examples given above: 

$ OP I I ON 

IOPT=l , 

IFITE=2 , 

IANAL=3 , 

IENG=1, 

$END 

$CONFIN 

DESRNG-750 .0, 

TWR=0 . 63, 

GW=30000.0, 0.0,28000.0, 35000.0, 
$END 


Execution 

TSW is executed from the UNIX command prompt. The syntax is: 

tsw -input InputFilename -output OutputFilename Subst itut ionFile 


The following rules apply: 

• The input file must exist 

• The substitution template file must exist 

• The input and output file must not have the same name 
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